-- Logs begin at Mon 2022-10-03 15:19:51 UTC, end at Mon 2022-10-03 16:15:57 UTC. -- Oct 03 15:44:29.682396 dv-cinder-844563-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 03 15:44:29.689658 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 03 15:44:29.691682 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 3 15:44:29 2022] *** Oct 03 15:44:29.691682 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 03 15:44:29.691682 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 03 15:44:29.691682 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: nodename: dv-cinder-844563-13-iscsi Oct 03 15:44:29.691682 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: machine: x86_64 Oct 03 15:44:29.691682 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: clock source: unix Oct 03 15:44:29.691682 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: pcre jit disabled Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: detected number of CPU cores: 4 Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: current working directory: / Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: detected binary path: /usr/bin/uwsgi-core Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: your processes number limit is 47680 Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: your memory page size is 4096 bytes Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: detected max file descriptor number: 2048 Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: lock engine: pthread robust mutexes Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: thunder lock: enabled Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 03 15:44:29.692120 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 03 15:44:29.733740 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: Python main interpreter initialized at 0x557f094ef9c0 Oct 03 15:44:29.733740 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: python threads support enabled Oct 03 15:44:29.733740 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: your server socket listen backlog is limited to 100 connections Oct 03 15:44:29.733740 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: your mercy for graceful operations on workers is 90 seconds Oct 03 15:44:29.734116 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: mapped 403077 bytes (393 KB) for 2 cores Oct 03 15:44:29.734204 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: *** Operational MODE: preforking *** Oct 03 15:44:29.734466 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: *** uWSGI is running in multiple interpreter mode *** Oct 03 15:44:29.734466 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: spawned uWSGI master process (pid: 81358) Oct 03 15:44:29.734667 dv-cinder-844563-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 03 15:44:29.735516 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: spawned uWSGI worker 1 (pid: 81367, cores: 1) Oct 03 15:44:29.735615 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: spawned uWSGI worker 2 (pid: 81368, cores: 1) Oct 03 15:44:29.737017 dv-cinder-844563-13-iscsi devstack@keystone.service[81358]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 03 15:44:34.086109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 15:44:34.086109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: from cryptography.utils import int_from_bytes Oct 03 15:44:34.086903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 15:44:34.086903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: from cryptography.utils import int_from_bytes Oct 03 15:44:34.087021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 15:44:34.087021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: from cryptography.utils import int_from_bytes Oct 03 15:44:34.087021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 15:44:34.087021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: from cryptography.utils import int_from_bytes Oct 03 15:44:34.137175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 15:44:34.137367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 15:44:34.137520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 15:44:34.137622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] command line args: [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 15:44:34.137724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 15:44:34.137809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 15:44:34.137920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] command line args: [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 15:44:34.137997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ================================================================================ {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 15:44:34.138193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 15:44:34.138272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] admin_token = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.138374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ================================================================================ {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 15:44:34.138451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] config_dir = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.138631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.138716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] admin_token = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.138815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] config_source = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.138897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] config_dir = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.138994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] debug = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] config_source = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] debug = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.139975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] list_limit = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] log_config_append = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] list_limit = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.140972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] log_config_append = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] log_dir = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] log_file = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] log_dir = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] log_file = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.141972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.142982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.143906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.144862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] publish_errors = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] publish_errors = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.145804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] standard_threads = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] standard_threads = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.146945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.147182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.147312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.147312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.147522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.147632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] use_journal = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.147632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] use_json = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.147878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] use_stderr = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] use_journal = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] use_syslog = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] use_json = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] use_stderr = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] use_syslog = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.148906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 15:44:34.149135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149718 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.149940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.150981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.151899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.152963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.153943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.154974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.155915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.156949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.157992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.158902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.159905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.160972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.161065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.161263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.161377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.161492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.161668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.161781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.161921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.162044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.162146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.162244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.162380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.162500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.162621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.163291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.163434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.163555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.163784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.163900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.164077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.164290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.164415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.164563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.164790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.164910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.165064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.165181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.165308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.165423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.165614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.165733 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.165879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.166077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.166203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.166351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.166534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.166658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.166820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.166975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.167101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.167302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.167443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.167560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.167773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.167965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.168993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.169947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.170131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.170304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.170429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.170607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.170812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.170930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.171095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.171283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.171410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.171582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.171740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.171857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.172049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.172210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.172337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.172513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.172674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.172806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.173930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.174954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.175921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] auth.external = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.176891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] auth.password = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] auth.token = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.177949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] auth.external = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.178068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.178182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.178328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] auth.password = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] auth.token = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] credential.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.179978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.180100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.180214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.180326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.180439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.180550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.180702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.180861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.181000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.181179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.181290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.181478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.181665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.181793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.181964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.182110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.182228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.182496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.182613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] credential.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.182734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.182897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.183895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.184009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.184143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.184324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.184443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.184646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.184853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.184973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.185162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.185289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.185452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.185703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.185820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.185993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.186193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.186320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.186464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.186581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.186694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.186806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.186948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.187069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.187193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.187421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.187540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.187654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.187901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.188032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.188149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.188329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.188456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.188609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.188838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.188961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.189118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.189308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.189426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.189602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.189794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.189912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.190081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.190248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.190366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.190536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.190662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.190777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.190897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.191888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.192993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.193134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.193253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.193432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.193631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.193758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.193947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.194070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.194306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.194431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.194546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.194772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.194891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.195025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.195283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.195401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.195536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.195770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.195889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.196068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.196233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.196352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.196549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.196708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.196844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.197967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.198944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.199997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.200995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.201922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.202951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.203990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.204935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.205982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.206097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.206248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.206362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.206475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.206587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.206780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.206899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.207930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.208903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.209907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.210991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.211919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] role.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.212984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] role.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.213981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.214913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] role.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.215948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.216126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.216553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.216817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.216939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] role.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.217074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.217194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.217321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.217566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.217818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.218052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.218309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.218435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.218552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.218788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.219166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.219280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.219437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.219564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.219701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.219817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.219958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.220208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.220448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.220687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.220823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.220937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221405 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.221862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.222893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.223949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.224061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.224194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.224578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.224747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.224868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.224982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.225107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.225742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.225944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.226089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.226342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.226462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.connection = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.226577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.226690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.226810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.226810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.227947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.228880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.229926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81367) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 15:44:34.230956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.230956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.231186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.231378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.231630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.231915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.232151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.232396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.232664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.232961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.connection = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.233212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.233463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.233706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.233950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.234191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.234431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.234671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.234910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.235157 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.235395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.235640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.235879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.236128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.236377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.236608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.236865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.237111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.237349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 15:44:34.237522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81368) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 15:44:34.260425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.260657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.260907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.263299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.263534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.263762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.269361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.272275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.298168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.298396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.298846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.299080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.353486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.354161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.402415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.402647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.403405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.403642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.562979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.563949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.564081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.564173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.564294 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.587757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.587983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588294 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.588995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.589123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.589204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.589352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.589542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.600676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.600960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.613418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.613525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.613602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.613727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.613822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.613954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81367) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.614814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81368) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 15:44:34.632619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.632857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 15:44:34.633104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.633187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.633318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 15:44:34.633401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 15:44:34.633654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.633736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.633853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 15:44:34.633934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.634169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.634307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.634390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.634463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.634637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.634896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.644001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.644283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.644534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.644615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.644841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.644926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.645119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.645405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.651909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.652759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.654852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.655137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.655420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.655677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.655758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.655965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.656047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.656224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.656306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.656495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.656576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.656790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.656885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.657070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.657150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.657344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.657424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.657615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.657697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.657892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.658169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.658440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.680994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.682027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.682465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.682748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.683502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.683790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.687429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.687715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.688000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.688499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.688814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.689099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.695275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.695749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.696239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.696532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.701465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.701761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.702046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.702232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.702315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.702530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.702620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.702823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.702904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.703112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.703191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.703380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.703463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.703659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.703933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.704204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.719482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.719764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.720356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.720641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.723650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.724827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.727226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.727525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.728303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.728603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.731742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.732887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.734909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.736040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.738144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.738447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.738734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.739019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.739307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.739608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.739687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.739943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.740235 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.740528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.741404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.741673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.755954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.756243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.756590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.756880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.757639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.757940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.758215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.758482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.763708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.765283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.765365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.766807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.766941 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.767085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.768461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.768883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.769812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.770101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.770389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.770680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.771068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.771562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.771852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.772786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.773086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.773473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.782772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.783059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.783342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.783623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.784004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.784296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.784574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.784653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.784956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.785246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.785533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.785921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.786223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.786509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.800099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.802167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.803024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.805765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.806725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.808216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.808686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.809710 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.809993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.810183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.810398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.811766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.812053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.812457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.816746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.818218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.818847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.820416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.821475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.821873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.822161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.822441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.823353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.823754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.824044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.824325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.830292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.830582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.830866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.832199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.832505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.833110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.838327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81367) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.838615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.838899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.839181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.839454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.839740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.840014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.840290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.840561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.840643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81368) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 15:44:34.840863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.840947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.841146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.841243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.841420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.841503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.841738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.841828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.842010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81367) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.842090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.842411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.842706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.842985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.843272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.843551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.843832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.844103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.844377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81368) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 15:44:34.877009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557f094ef9c0 pid: 81367 (default app) Oct 03 15:44:34.880273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fec932-3c09-42a6-83b8-ef46fe9feb3b None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:34.880559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fec932-3c09-42a6-83b8-ef46fe9feb3b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:34.880858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fec932-3c09-42a6-83b8-ef46fe9feb3b None None] PATH_INFO: `/v3` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:34.883208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557f094ef9c0 pid: 81368 (default app) Oct 03 15:44:34.885659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Mon Oct 3 15:44:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:41.658277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760a36-2221-4d7b-8ae7-a4fa3f4ab285 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:41.658770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760a36-2221-4d7b-8ae7-a4fa3f4ab285 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:41.658770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97760a36-2221-4d7b-8ae7-a4fa3f4ab285 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:41.663547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:44:41.668869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:41.669318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:41.669457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:41.743823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: INFO dbcounter [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] Registered counter for database keystone Oct 03 15:44:41.762636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG oslo_db.sqlalchemy.engines [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81367) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 15:44:41.768816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writer thread running {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 15:44:41.954949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG passlib.handlers.bcrypt [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81367) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 15:44:41.955260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG passlib.handlers.bcrypt [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81367) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 15:44:42.013548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:42.015142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:42.143953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 03 15:44:42.597134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97760a36-2221-4d7b-8ae7-a4fa3f4ab285 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:42.667785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: INFO dbcounter [None req-97760a36-2221-4d7b-8ae7-a4fa3f4ab285 None None] Registered counter for database keystone Oct 03 15:44:42.689134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG oslo_db.sqlalchemy.engines [None req-97760a36-2221-4d7b-8ae7-a4fa3f4ab285 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81368) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 15:44:42.695331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writer thread running {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 15:44:42.814081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97760a36-2221-4d7b-8ae7-a4fa3f4ab285 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:42.818343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89673f-a178-4e59-b90a-b3cb36f80bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:42.819056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89673f-a178-4e59-b90a-b3cb36f80bea admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:42.819395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89673f-a178-4e59-b90a-b3cb36f80bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:42.819695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89673f-a178-4e59-b90a-b3cb36f80bea admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:42.827774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89673f-a178-4e59-b90a-b3cb36f80bea admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:42.879555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bf89673f-a178-4e59-b90a-b3cb36f80bea admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 15:44:42.879837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89673f-a178-4e59-b90a-b3cb36f80bea admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:42.889578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Oct 3 15:44:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:44:42.895475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:42.913676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72497066-a80e-4dff-87ed-11ad0401e911 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:42.918043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd8c5a-5edf-4282-a6ef-ffa0b46641b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:42.919143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd8c5a-5edf-4282-a6ef-ffa0b46641b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:42.919416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd8c5a-5edf-4282-a6ef-ffa0b46641b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:42.919679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd8c5a-5edf-4282-a6ef-ffa0b46641b8 admin admin] PATH_INFO: `/v3/projects/0727be3aa6194ac280b9115cbd69b50d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:42.921291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd8c5a-5edf-4282-a6ef-ffa0b46641b8 admin admin] RBAC: Authorizing `identity:get_project(project_id=0727be3aa6194ac280b9115cbd69b50d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:42.972744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING keystone.common.rbac_enforcer.enforcer [None req-47fd8c5a-5edf-4282-a6ef-ffa0b46641b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 15:44:42.973105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd8c5a-5edf-4282-a6ef-ffa0b46641b8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:42.978456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Mon Oct 3 15:44:42 2022] GET /identity/v3/projects/0727be3aa6194ac280b9115cbd69b50d => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:45.086344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17736fb-e676-4b8e-967e-c6716f65f6fb None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:45.086885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17736fb-e676-4b8e-967e-c6716f65f6fb None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:45.086885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17736fb-e676-4b8e-967e-c6716f65f6fb None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:45.090985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:45.096431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae0b41-b96a-4de0-a04f-050f5130ffaa None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:45.096826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae0b41-b96a-4de0-a04f-050f5130ffaa None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:45.097001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae0b41-b96a-4de0-a04f-050f5130ffaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:45.117701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c7ae0b41-b96a-4de0-a04f-050f5130ffaa None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:45.119198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c7ae0b41-b96a-4de0-a04f-050f5130ffaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:45.126964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 03 15:44:45.401801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17736fb-e676-4b8e-967e-c6716f65f6fb None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:45.414472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17736fb-e676-4b8e-967e-c6716f65f6fb None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:45.418700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a4a09-04fb-4929-aff7-4fd0db4cf0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:45.419299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a4a09-04fb-4929-aff7-4fd0db4cf0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:45.419576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a4a09-04fb-4929-aff7-4fd0db4cf0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:45.419845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a4a09-04fb-4929-aff7-4fd0db4cf0ee admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:45.427420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a4a09-04fb-4929-aff7-4fd0db4cf0ee admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:45.430080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a4a09-04fb-4929-aff7-4fd0db4cf0ee admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:45.438688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Oct 3 15:44:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:44:45.444189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae0b41-b96a-4de0-a04f-050f5130ffaa None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:45.453286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae0b41-b96a-4de0-a04f-050f5130ffaa None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:45.457667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65349070-fc70-4a8e-9c7b-7c8633b49b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:45.458287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65349070-fc70-4a8e-9c7b-7c8633b49b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:45.458569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65349070-fc70-4a8e-9c7b-7c8633b49b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:45.458844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65349070-fc70-4a8e-9c7b-7c8633b49b83 admin admin] PATH_INFO: `/v3/users/272aa82f64454c7aaff95459c4c29980` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:45.460179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65349070-fc70-4a8e-9c7b-7c8633b49b83 admin admin] RBAC: Authorizing `identity:get_user(user_id=272aa82f64454c7aaff95459c4c29980)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:45.462071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65349070-fc70-4a8e-9c7b-7c8633b49b83 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:45.463590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Mon Oct 3 15:44:45 2022] GET /identity/v3/users/272aa82f64454c7aaff95459c4c29980 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:47.707368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34ecb8-3c6e-4b06-9de2-6dc08fa4d873 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:47.707876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34ecb8-3c6e-4b06-9de2-6dc08fa4d873 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:47.707876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34ecb8-3c6e-4b06-9de2-6dc08fa4d873 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:47.709290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:47.714171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:47.714390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb2ab-fd1a-4abb-a984-a2199895401e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:47.714781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:47.714883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb2ab-fd1a-4abb-a984-a2199895401e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:47.714981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:47.715068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fb2ab-fd1a-4abb-a984-a2199895401e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:47.716217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:44:47.721201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689791-69e4-470f-af9a-b4cc7637bbb1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:47.721478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689791-69e4-470f-af9a-b4cc7637bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:47.721746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689791-69e4-470f-af9a-b4cc7637bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:47.740211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1d689791-69e4-470f-af9a-b4cc7637bbb1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:47.742074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1d689791-69e4-470f-af9a-b4cc7637bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:47.757702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG passlib.handlers.bcrypt [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81368) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 15:44:47.758085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG passlib.handlers.bcrypt [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81368) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 15:44:47.807339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:47.808939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:47.821741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 7/12] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:44:47.908923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 6/13] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:44:48.085528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689791-69e4-470f-af9a-b4cc7637bbb1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.110532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:48.173585 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.190388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689791-69e4-470f-af9a-b4cc7637bbb1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.228495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.229170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.229448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.229714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.240679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:44:48.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:44:48.250553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:44:48.251614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 8/14] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:44:48.256371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.281811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3daa64-8aa9-48fc-98a6-35afccad07d5 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.284595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:48.321611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.322489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.322764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.323041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.338854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:48.357887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562629fa-ef17-4a48-b927-33ffaa2439ad None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.391540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:48.391706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.392204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.392483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.392781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.393083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:48.393483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:44:48.393807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:48.395202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:48.407143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 9/15] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:44:48.413692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.490233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 7/16] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 15:44:48.496477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.499480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:48.513615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034553-6da8-4732-8da1-7f2dde43bd8a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.553671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.554600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.554884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.555159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] PATH_INFO: `/v3/users/272aa82f64454c7aaff95459c4c29980` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.556345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] RBAC: Authorizing `identity:get_user(user_id=272aa82f64454c7aaff95459c4c29980)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:48.557878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:48.559473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 10/17] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/users/272aa82f64454c7aaff95459c4c29980 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:44:48.564985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.579776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:48.594574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f8ac6-1127-4178-ba21-ba87a8a1851c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.628378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.629297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.629575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.629850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.647941 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:48.649452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:48.665111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb8e9c-bd9e-4906-b298-af9baba290b0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:44:48.668822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:44:48.669851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:44:48.675526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:44:48.675526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:44:48.675526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:44:48.675526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 8/18] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:44:48.675783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.703171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.703803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.704082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.704351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.705513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:48.706989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:48.708519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 11/19] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:44:48.714265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:48.733432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:48.779231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019187b-5677-4b3d-b2d9-c45378368bd8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.795594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:48.812487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67c7d7-0b66-4186-bc61-8ab6b9efeb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.813125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67c7d7-0b66-4186-bc61-8ab6b9efeb21 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.813402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67c7d7-0b66-4186-bc61-8ab6b9efeb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.813672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67c7d7-0b66-4186-bc61-8ab6b9efeb21 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.814310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67c7d7-0b66-4186-bc61-8ab6b9efeb21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:44:48.814612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67c7d7-0b66-4186-bc61-8ab6b9efeb21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:48.814754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdde450-4352-472d-9b80-dee6b0f06aaa None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:48.816009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67c7d7-0b66-4186-bc61-8ab6b9efeb21 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:48.832700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 9/20] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:44:48.849101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71848cbf-8955-43bb-9dba-f971701282ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:48.850561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71848cbf-8955-43bb-9dba-f971701282ea None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:48.850836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71848cbf-8955-43bb-9dba-f971701282ea None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:48.851107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71848cbf-8955-43bb-9dba-f971701282ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:48.851868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71848cbf-8955-43bb-9dba-f971701282ea None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.domain.id=default, user.id=272aa82f64454c7aaff95459c4c29980) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:44:48.852172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71848cbf-8955-43bb-9dba-f971701282ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:48.853692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71848cbf-8955-43bb-9dba-f971701282ea None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:48.863270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 12/21] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Oct 3 15:44:48 2022] GET /identity/v3/role_assignments?user.id=272aa82f64454c7aaff95459c4c29980&scope.domain.id=default&role.id=0cb19d985d3d456983e0b1624610b058 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:44:51.279245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c47d44-3db9-44cd-bb7a-90665733990e None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:51.280139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c47d44-3db9-44cd-bb7a-90665733990e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:51.280549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c47d44-3db9-44cd-bb7a-90665733990e None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:51.282047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 10/22] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:51.288115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00df3a-00fc-4c6b-b64d-b9bdc384d0e0 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:51.289170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00df3a-00fc-4c6b-b64d-b9bdc384d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:51.289604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00df3a-00fc-4c6b-b64d-b9bdc384d0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:51.316509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1f00df3a-00fc-4c6b-b64d-b9bdc384d0e0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:51.318025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1f00df3a-00fc-4c6b-b64d-b9bdc384d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:51.411495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 13/23] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:44:51.556253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52dad-83fe-4567-8c34-4a7a1ca6b898 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:51.556497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52dad-83fe-4567-8c34-4a7a1ca6b898 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:51.556750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52dad-83fe-4567-8c34-4a7a1ca6b898 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:51.558048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 11/24] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:51.563528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29c411-c7f5-441e-96f0-64fdcbf5ceab None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:51.564046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29c411-c7f5-441e-96f0-64fdcbf5ceab None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:51.564525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29c411-c7f5-441e-96f0-64fdcbf5ceab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:51.589758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ad29c411-c7f5-441e-96f0-64fdcbf5ceab None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:51.591337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ad29c411-c7f5-441e-96f0-64fdcbf5ceab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:51.663338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 14/25] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:44:51.763597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf52dad-83fe-4567-8c34-4a7a1ca6b898 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:51.901495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf52dad-83fe-4567-8c34-4a7a1ca6b898 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:51.922029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:51.943804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cc314-7d0a-4832-8849-6a27f62854a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:51.944619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cc314-7d0a-4832-8849-6a27f62854a5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:51.944923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cc314-7d0a-4832-8849-6a27f62854a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:51.945191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cc314-7d0a-4832-8849-6a27f62854a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:51.960063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cc314-7d0a-4832-8849-6a27f62854a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:51.961751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cc314-7d0a-4832-8849-6a27f62854a5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-749cc314-7d0a-4832-8849-6a27f62854a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:44:51.983191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:44:51.984111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:44:51.984925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:44:51.984925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:44:51.984925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:44:51.984925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 12/26] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:44:51.992412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29c411-c7f5-441e-96f0-64fdcbf5ceab None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:52.001526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:52.047205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37dcc-780f-45cd-bd67-0e0a1899de76 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.047518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37dcc-780f-45cd-bd67-0e0a1899de76 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.047754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc37dcc-780f-45cd-bd67-0e0a1899de76 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.049466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 13/27] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:52.054998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d0b2a-59a8-42b8-a3d0-b93c0d5b8785 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.055276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d0b2a-59a8-42b8-a3d0-b93c0d5b8785 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.055545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d0b2a-59a8-42b8-a3d0-b93c0d5b8785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.085544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-675d0b2a-59a8-42b8-a3d0-b93c0d5b8785 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:52.087641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-675d0b2a-59a8-42b8-a3d0-b93c0d5b8785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:52.170615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29c411-c7f5-441e-96f0-64fdcbf5ceab None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:52.180955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 14/28] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:44:52.182426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d0b2a-59a8-42b8-a3d0-b93c0d5b8785 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:52.234760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11170989-9274-423e-bde2-ee135927ffea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:52.235770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11170989-9274-423e-bde2-ee135927ffea None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.236241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11170989-9274-423e-bde2-ee135927ffea None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.237785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11170989-9274-423e-bde2-ee135927ffea None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.238629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11170989-9274-423e-bde2-ee135927ffea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:44:52.239828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11170989-9274-423e-bde2-ee135927ffea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:52.242054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11170989-9274-423e-bde2-ee135927ffea None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:52.269596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 15/29] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:52.271197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a370b8-97ba-45b9-a5ed-2a9ce84ebe08 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.271701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a370b8-97ba-45b9-a5ed-2a9ce84ebe08 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.272193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a370b8-97ba-45b9-a5ed-2a9ce84ebe08 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.273712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 16/30] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:52.275034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a370b8-97ba-45b9-a5ed-2a9ce84ebe08 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:52.348401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d0b2a-59a8-42b8-a3d0-b93c0d5b8785 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:52.353031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:52.378673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:52.398524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4394e2-b95d-4616-9181-bc468cd25f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:52.400121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4394e2-b95d-4616-9181-bc468cd25f92 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.400591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4394e2-b95d-4616-9181-bc468cd25f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.401206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4394e2-b95d-4616-9181-bc468cd25f92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.415804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4394e2-b95d-4616-9181-bc468cd25f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-fa4394e2-b95d-4616-9181-bc468cd25f92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:44:52.423876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:44:52.424868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:44:52.425636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:44:52.428561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 15/31] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:44:52.430045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054a85-a9aa-418b-b554-c8eb93b0acfa None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.430513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054a85-a9aa-418b-b554-c8eb93b0acfa None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.431275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054a85-a9aa-418b-b554-c8eb93b0acfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.452168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-80054a85-a9aa-418b-b554-c8eb93b0acfa None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:52.453704 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-80054a85-a9aa-418b-b554-c8eb93b0acfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:52.480110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a370b8-97ba-45b9-a5ed-2a9ce84ebe08 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:52.528839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:52.529506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.529794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.530128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.531339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:52.533448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:52.539501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 16/32] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:44:52.540607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80054a85-a9aa-418b-b554-c8eb93b0acfa None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:52.680448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:52.703012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80054a85-a9aa-418b-b554-c8eb93b0acfa None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:52.761330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:52.761951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.762228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.762499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.763233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:44:52.763537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:52.764955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:52.778972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 17/33] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:52.779869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:52.786872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 17/34] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:44:52 2022] POST /identity/v3/projects => generated 301 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:44:52.787862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:52.938232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855183-b9e6-404e-ac75-1903ef0e48be None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:52.955167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764dea0-5480-4784-b904-e950350886e0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:52.975860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:52.986190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:52.986810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:52.987093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:52.987359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:52.988033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:52.990149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:53.005727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faa617-fdce-47d7-865c-ddc328260c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:53.007147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa617-fdce-47d7-865c-ddc328260c58 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.008880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa617-fdce-47d7-865c-ddc328260c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.008880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa617-fdce-47d7-865c-ddc328260c58 None admin] PATH_INFO: `/v3/users/272aa82f64454c7aaff95459c4c29980` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.009852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa617-fdce-47d7-865c-ddc328260c58 None admin] RBAC: Authorizing `identity:get_user(user_id=272aa82f64454c7aaff95459c4c29980)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:53.011835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa617-fdce-47d7-865c-ddc328260c58 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:53.013759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 18/35] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:44:52 2022] GET /identity/v3/users/272aa82f64454c7aaff95459c4c29980 => generated 247 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:53.014556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629270a-c270-41d7-8308-313d0d42de47 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.014840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629270a-c270-41d7-8308-313d0d42de47 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.015117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629270a-c270-41d7-8308-313d0d42de47 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.016514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 19/36] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:53.021573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629270a-c270-41d7-8308-313d0d42de47 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:53.133663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 18/37] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 15:44:52 2022] POST /identity/v3/roles => generated 219 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:44:53.134817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:53.155877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:53.271185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c091aa-c573-4232-a431-3c34c1ab6967 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:53.275811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629270a-c270-41d7-8308-313d0d42de47 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:53.322954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:53.324460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.324950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.325343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe9e39-005c-4a05-9781-0f3ffdf7d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:53.325557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.326177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9e39-005c-4a05-9781-0f3ffdf7d179 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.326536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9e39-005c-4a05-9781-0f3ffdf7d179 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.326718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:53.326987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9e39-005c-4a05-9781-0f3ffdf7d179 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.328441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:53.328693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe9e39-005c-4a05-9781-0f3ffdf7d179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:53.330384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe9e39-005c-4a05-9781-0f3ffdf7d179 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:53.332385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 19/38] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:53.333937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbbbb-a575-44bc-92c1-004c8e39a30b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.335399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbbbb-a575-44bc-92c1-004c8e39a30b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.335399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bbbbb-a575-44bc-92c1-004c8e39a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.354441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 20/39] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 15:44:53 2022] POST /identity/v3/roles => generated 225 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:44:53.355293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:53.358238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-171bbbbb-a575-44bc-92c1-004c8e39a30b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:53.359926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-171bbbbb-a575-44bc-92c1-004c8e39a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:53.430471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:53.460687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 20/40] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:44:53.527733 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22e570-3e66-4e9f-acb6-1c9a2f30f036 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:53.547374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c569-91e5-4718-97fd-68f47846f447 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.547684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c569-91e5-4718-97fd-68f47846f447 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.547933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83c569-91e5-4718-97fd-68f47846f447 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.549490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 21/41] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:44:53.555147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da036e2-8497-480d-967b-53d63dc04cf1 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.555437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da036e2-8497-480d-967b-53d63dc04cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.555712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da036e2-8497-480d-967b-53d63dc04cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.568388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:53.569480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.569480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.569770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] PATH_INFO: `/v3/domains/default/users/272aa82f64454c7aaff95459c4c29980/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.571935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=272aa82f64454c7aaff95459c4c29980, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:53.573874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:53.578578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7da036e2-8497-480d-967b-53d63dc04cf1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:53.580838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7da036e2-8497-480d-967b-53d63dc04cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:53.582159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:53.590339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 21/42] 10.222.0.33 () {58 vars in 1361 bytes} [Mon Oct 3 15:44:53 2022] PUT /identity/v3/domains/default/users/272aa82f64454c7aaff95459c4c29980/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:44:53.676840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 22/43] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:44:53.779960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:53.884900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c065-4081-49a6-b288-2811f46ecbaa None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:53.887239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:53.922987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:53.923603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:53.923882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:53.924151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:53.924874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:53.926268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:53.941637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 22/44] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 15:44:53 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:44:53.952967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da036e2-8497-480d-967b-53d63dc04cf1 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:54.067829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da036e2-8497-480d-967b-53d63dc04cf1 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:54.106812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e0263-f075-4bae-88b4-796e0e1bec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:54.107961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e0263-f075-4bae-88b4-796e0e1bec15 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:54.107961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e0263-f075-4bae-88b4-796e0e1bec15 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:54.108127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e0263-f075-4bae-88b4-796e0e1bec15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:54.109162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e0263-f075-4bae-88b4-796e0e1bec15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:54.111074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e0263-f075-4bae-88b4-796e0e1bec15 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:54.112365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 23/45] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:54.119662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:54.229190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648cc07-f2fa-4425-8ddd-82030865d68c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:54.261632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102cba3-e4e7-468a-81a2-f75655c6e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:54.262232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102cba3-e4e7-468a-81a2-f75655c6e519 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:54.262511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102cba3-e4e7-468a-81a2-f75655c6e519 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:54.262779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102cba3-e4e7-468a-81a2-f75655c6e519 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:54.263437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102cba3-e4e7-468a-81a2-f75655c6e519 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:54.264925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102cba3-e4e7-468a-81a2-f75655c6e519 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:54.292344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 23/46] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:44:54 2022] POST /identity/v3/projects => generated 312 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:44:55.801576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e477fba-3d31-4bf5-b812-a69bec96d4d3 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:55.802065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e477fba-3d31-4bf5-b812-a69bec96d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:55.802065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e477fba-3d31-4bf5-b812-a69bec96d4d3 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:55.803702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 24/47] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:55.808906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688b11e-e406-4422-83ad-ccb686eea6bb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:55.809315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688b11e-e406-4422-83ad-ccb686eea6bb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:55.809465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688b11e-e406-4422-83ad-ccb686eea6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:55.827683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-b688b11e-e406-4422-83ad-ccb686eea6bb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:55.829230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b688b11e-e406-4422-83ad-ccb686eea6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:55.840765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:55.899766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 24/48] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:44:56.170796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e477fba-3d31-4bf5-b812-a69bec96d4d3 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:56.180358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:56.379209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e477fba-3d31-4bf5-b812-a69bec96d4d3 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:56.413366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b061a-bd60-4475-866e-0e9d6cf37cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:56.413998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b061a-bd60-4475-866e-0e9d6cf37cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:56.414278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b061a-bd60-4475-866e-0e9d6cf37cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:56.414552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b061a-bd60-4475-866e-0e9d6cf37cb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:56.424467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b061a-bd60-4475-866e-0e9d6cf37cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-b42b061a-bd60-4475-866e-0e9d6cf37cb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:44:56.429835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:44:56.430649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:44:56.431393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 25/49] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:44:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:44:56.436789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688b11e-e406-4422-83ad-ccb686eea6bb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:56.468417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e514-52fb-400f-a08d-9f94ae3a1e18 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:56.468699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e514-52fb-400f-a08d-9f94ae3a1e18 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:56.469006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1e514-52fb-400f-a08d-9f94ae3a1e18 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:56.470342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 26/50] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:44:56.476280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422977-3fa5-49df-adc5-a66d569192ed None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:56.476564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422977-3fa5-49df-adc5-a66d569192ed None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:56.476881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af422977-3fa5-49df-adc5-a66d569192ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:56.494584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-af422977-3fa5-49df-adc5-a66d569192ed None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:56.495792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-af422977-3fa5-49df-adc5-a66d569192ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:56.535951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688b11e-e406-4422-83ad-ccb686eea6bb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:56.554051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:56.561915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 27/51] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:44:56.570988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:56.571599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:56.571976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:56.572253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:56.572971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:44:56.573276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:56.574661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:56.582858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 25/52] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:56.589148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af422977-3fa5-49df-adc5-a66d569192ed None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:56.685312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af422977-3fa5-49df-adc5-a66d569192ed None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:56.719215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:56.719852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:56.720133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:56.720407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] PATH_INFO: `/v3/users/272aa82f64454c7aaff95459c4c29980` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:56.721951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] RBAC: Authorizing `identity:get_user(user_id=272aa82f64454c7aaff95459c4c29980)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:56.723424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:56.725324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 28/53] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:44:56 2022] GET /identity/v3/users/272aa82f64454c7aaff95459c4c29980 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:56.730879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:56.742010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:56.830068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f8d7c-3ad7-46c4-89db-96c351e61dcc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:56.845994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:56.864397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:56.865095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:56.865395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:56.865719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:56.866900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:56.868382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:56.870003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 26/54] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:56.875641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:56.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:56.952878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:56.953179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d0bb7-e102-461e-82e3-b7d9c40709b5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:56.976442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7a95e-c05e-4f46-b60f-889a7f7a1443 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:56.989917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:56.991090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:56.991365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:56.991631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:56.992781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:56.994244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:56.995809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 29/55] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:44:57.001302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:57.010297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edb38f-ae32-4409-b943-0b6da26f9b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:57.010900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edb38f-ae32-4409-b943-0b6da26f9b76 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:57.011180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edb38f-ae32-4409-b943-0b6da26f9b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:57.011451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edb38f-ae32-4409-b943-0b6da26f9b76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:57.012177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edb38f-ae32-4409-b943-0b6da26f9b76 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.domain.id=default, user.id=272aa82f64454c7aaff95459c4c29980) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:44:57.012482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edb38f-ae32-4409-b943-0b6da26f9b76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:57.013980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edb38f-ae32-4409-b943-0b6da26f9b76 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:57.019573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 27/56] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Oct 3 15:44:56 2022] GET /identity/v3/role_assignments?user.id=272aa82f64454c7aaff95459c4c29980&scope.domain.id=default&role.id=0cb19d985d3d456983e0b1624610b058 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:57.109135 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b40ed-7042-420b-817a-8ec04c440c14 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:57.140497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:57.145931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91bfd9-b30b-4400-9105-b9ecb092ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:57.146962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91bfd9-b30b-4400-9105-b9ecb092ca34 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:57.147243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91bfd9-b30b-4400-9105-b9ecb092ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:57.147511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91bfd9-b30b-4400-9105-b9ecb092ca34 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:57.148176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91bfd9-b30b-4400-9105-b9ecb092ca34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:57.149978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91bfd9-b30b-4400-9105-b9ecb092ca34 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:57.180341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 30/57] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Oct 3 15:44:57 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:44:59.337974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc14475-8e6c-4894-a665-a23344089df7 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:59.338408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc14475-8e6c-4894-a665-a23344089df7 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:59.338521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc14475-8e6c-4894-a665-a23344089df7 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:59.339858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 28/58] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:44:59.345122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980efda9-d438-4e9b-90eb-d298fa2cecc2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:59.345536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980efda9-d438-4e9b-90eb-d298fa2cecc2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:59.345683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980efda9-d438-4e9b-90eb-d298fa2cecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:59.364372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-980efda9-d438-4e9b-90eb-d298fa2cecc2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:44:59.365890 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-980efda9-d438-4e9b-90eb-d298fa2cecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:44:59.429988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 31/59] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:44:59.701974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc14475-8e6c-4894-a665-a23344089df7 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:59.801566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc14475-8e6c-4894-a665-a23344089df7 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:59.835569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1333f57-c7ed-4a89-9de1-23f1c158a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:59.836180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1333f57-c7ed-4a89-9de1-23f1c158a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:59.836457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1333f57-c7ed-4a89-9de1-23f1c158a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:59.836751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1333f57-c7ed-4a89-9de1-23f1c158a0b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:59.837858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1333f57-c7ed-4a89-9de1-23f1c158a0b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:59.839312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1333f57-c7ed-4a89-9de1-23f1c158a0b1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:44:59.840857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 29/60] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:44:59.846251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980efda9-d438-4e9b-90eb-d298fa2cecc2 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:44:59.947231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980efda9-d438-4e9b-90eb-d298fa2cecc2 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:44:59.961243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:44:59.981891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a6efb-5bcc-4e79-ab9e-436d6f085a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:44:59.982499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6efb-5bcc-4e79-ab9e-436d6f085a30 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:44:59.982785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6efb-5bcc-4e79-ab9e-436d6f085a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:44:59.983056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6efb-5bcc-4e79-ab9e-436d6f085a30 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:44:59.983701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a6efb-5bcc-4e79-ab9e-436d6f085a30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:44:59.985175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a6efb-5bcc-4e79-ab9e-436d6f085a30 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:00.011489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 32/61] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:44:59 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:03.440802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b2012-563c-4e8b-b809-65d24456a7ac None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.441696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b2012-563c-4e8b-b809-65d24456a7ac None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.442157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b2012-563c-4e8b-b809-65d24456a7ac None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.444305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 30/62] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:03.449786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc924a6b-5e36-4304-bc0f-ea27ed3fd042 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.450200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc924a6b-5e36-4304-bc0f-ea27ed3fd042 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.450415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc924a6b-5e36-4304-bc0f-ea27ed3fd042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.477340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253a3cf-0302-4603-bcf8-eac87367b9b4 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.477980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253a3cf-0302-4603-bcf8-eac87367b9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.478457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253a3cf-0302-4603-bcf8-eac87367b9b4 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.478908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-cc924a6b-5e36-4304-bc0f-ea27ed3fd042 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:03.480117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cc924a6b-5e36-4304-bc0f-ea27ed3fd042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:03.480647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 31/63] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:03.487657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3966751-bb00-4670-be34-5c5ca289e114 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.488212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3966751-bb00-4670-be34-5c5ca289e114 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.488684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3966751-bb00-4670-be34-5c5ca289e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.525768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-b3966751-bb00-4670-be34-5c5ca289e114 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:03.526984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b3966751-bb00-4670-be34-5c5ca289e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:03.529424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:03.569268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 33/64] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:03.571113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60759e3-9697-4e1b-8c2f-5f00fd920a22 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.571599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60759e3-9697-4e1b-8c2f-5f00fd920a22 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.572089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60759e3-9697-4e1b-8c2f-5f00fd920a22 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.573581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 34/65] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:03.586134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75640e8e-52bd-42c7-8052-44b6b01aba93 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.586700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75640e8e-52bd-42c7-8052-44b6b01aba93 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.587167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75640e8e-52bd-42c7-8052-44b6b01aba93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.617243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-75640e8e-52bd-42c7-8052-44b6b01aba93 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:03.618880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-75640e8e-52bd-42c7-8052-44b6b01aba93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:03.641247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 32/66] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:03.724107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 35/67] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 03 15:45:03.812711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d37243-7b7c-495e-b4eb-4bbf8a86a5c6 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.813384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d37243-7b7c-495e-b4eb-4bbf8a86a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.813858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d37243-7b7c-495e-b4eb-4bbf8a86a5c6 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.815620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 33/68] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:03.821476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b0d1-f0db-4f34-8a04-f2fccb61e377 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:03.821702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b0d1-f0db-4f34-8a04-f2fccb61e377 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:03.821978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b0d1-f0db-4f34-8a04-f2fccb61e377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:03.850530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f860b0d1-f0db-4f34-8a04-f2fccb61e377 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:03.852071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f860b0d1-f0db-4f34-8a04-f2fccb61e377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:03.885467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:03.954059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 36/69] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:04.006058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d37243-7b7c-495e-b4eb-4bbf8a86a5c6 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.018478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbac89-bd38-4ed3-bdbe-6a5fb0ad3e7f None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.018765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbac89-bd38-4ed3-bdbe-6a5fb0ad3e7f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.019043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbac89-bd38-4ed3-bdbe-6a5fb0ad3e7f None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.020394 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 37/70] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:04.027646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6d218-036a-4cd5-9389-be56f2f7c34e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.027947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6d218-036a-4cd5-9389-be56f2f7c34e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.028202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6d218-036a-4cd5-9389-be56f2f7c34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.052489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-82f6d218-036a-4cd5-9389-be56f2f7c34e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:04.053730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-82f6d218-036a-4cd5-9389-be56f2f7c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:04.126134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d37243-7b7c-495e-b4eb-4bbf8a86a5c6 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.126797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 38/71] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:04.127544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6d218-036a-4cd5-9389-be56f2f7c34e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.178145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.179652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.180169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.180643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.191651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.200095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:04.205247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:04.211800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 34/72] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:04.213145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.235601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.254647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6d218-036a-4cd5-9389-be56f2f7c34e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.293274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.293913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.294193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.294463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.308694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:04.313561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:04.314399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:04.315159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 39/73] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:04.315767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.337639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63136cd6-93fd-4225-832a-1667b21803a3 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.373716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.374654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.375081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.375499 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.387666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:04.392448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:04.393294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:04.394056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 35/74] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:04.394690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.432261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddd015-cc46-400e-b663-c0ad8996aa73 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.453178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.466489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.467095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.467375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.467648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.468312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:04.468617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:04.470051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:04.478799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 40/75] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:04.479598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.495132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.500332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd88b7a-ab40-4187-bce6-a749ce6ff2c6 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.542416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.543066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.543345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.543621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.556180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:04.561255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:04.562103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:04.562877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 36/76] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:04.563501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.587536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879369-950c-47d0-8987-b5797fe5c987 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.621176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.621786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.622069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.622388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.623051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:04.623359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:04.624840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:04.633722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 41/77] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:04.634525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.666800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56948d12-df69-497c-a787-be7402d7bc99 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.684069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.701974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.702582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.702860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.703131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.703797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:04.704099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:04.705560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:04.710569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.714190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 37/78] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:04.714996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.736070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fc9cd-0280-4272-a1eb-a88fe1b4339a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.770646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.771282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.771559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.771829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.772971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:04.774410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:04.775960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 42/79] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:04.776740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.815817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c00109-c2d2-41a1-8edd-31872856a809 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.853203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.853832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.854111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.854384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.869361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:04.870853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:04.872507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 38/80] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:04.873310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.882130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9340b7-fd5f-4f3e-89ee-7f3a3ec09cad None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:04.913662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.915881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:04.916502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:04.916900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:04.917095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:04.917789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:04.918097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:04.919485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:04.928249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 43/81] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:04.929141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:04.934397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:04.975688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e1579-5440-43c7-8323-bf49a098cccf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.009414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.010047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.010325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.010599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.011731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.013217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.014782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 39/82] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:05.017516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:05.029661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d97a2f-5654-46ab-816d-e6ee9ef59c0e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.063081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.063709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.063987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.064254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] PATH_INFO: `/v3/users/272aa82f64454c7aaff95459c4c29980` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.065389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=272aa82f64454c7aaff95459c4c29980)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.066817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.068416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 44/83] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:04 2022] GET /identity/v3/users/272aa82f64454c7aaff95459c4c29980 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:05.069298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:05.121825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acac28-e79a-4852-b8b8-cd51fc13cd6c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.128684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.136614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.156043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.156654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.157125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.157416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.161488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.163538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.169258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f28139-8986-4799-b2d1-7d51772dc1c4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.206933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 40/84] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:45:05 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:05.206933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:05.217924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.219323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.219613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.219893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.305474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.307277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.314983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc595ef-6d1c-4f56-b0e8-f9079a690588 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.354874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.355535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.355815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.356085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.357331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.358844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.360482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 41/85] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:05 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:05.361312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:05.401335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.407593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.439053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 45/86] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:05.439053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:05.500804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002216a-5e54-4a68-a8d0-2c51fd653879 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.554738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.555353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.555631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.555892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.564810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a6237-fab3-483a-adff-4c476237f9c3 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.615601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.616503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.617010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.617428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.640130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.641841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.667821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.676127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.707501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.709888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.754026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 42/87] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:05.754388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:05.832779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 46/88] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:05.834343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:05.862608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a480-180a-41bd-89c9-fa00134bd24e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.893850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.908753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.908753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.908753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.908753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.933720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:05.952486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0090d-c49b-4be3-8604-e268403b10d0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:05.988006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:05.988568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:05.988898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:05.989177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:05.990848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.992322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.992591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:05.994179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 47/89] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:05 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:05.994412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:05.994860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:06.106040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 43/90] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:06.106674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:06.135969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d77994-7f38-472f-aba6-6073f4fcd3b2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:06.141333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:06.186523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:06.187416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:06.188558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:06.188898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:06.190049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:06.191505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:06.194192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 48/91] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:05 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:06.194998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:06.221114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:06.226073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda93f3-dc1b-49df-8796-c7991e31e540 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:06.265905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:06.266476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:06.266762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:06.267036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:06.268254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:06.269795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:06.277823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 44/92] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:06 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:06.278736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:06.332316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f892-3d52-43e2-8a58-2789e879ac31 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:06.375003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:06.375612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:06.375898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:06.376175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:06.377146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] RBAC: Adding query filter params (role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=c037029a4d494bcab2968cdb7f9a9917) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:06.377470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:06.378940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:06.388230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 49/93] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:06 2022] GET /identity/v3/role_assignments?user.id=c037029a4d494bcab2968cdb7f9a9917&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:06.389285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:06.408627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:06.414438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6515eb-fd01-4a1b-b73e-4458931e53e9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:06.454237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:06.455274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:06.455771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:06.456301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] PATH_INFO: `/v3/projects/1807b59758d54d56a02fa69faa896ef0` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:06.457697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] RBAC: Authorizing `identity:get_project(project_id=1807b59758d54d56a02fa69faa896ef0)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:06.459585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:06.461472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 45/94] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:06 2022] GET /identity/v3/projects/1807b59758d54d56a02fa69faa896ef0 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:06.462549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:06.478586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:06.516598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342da577-fa32-43c7-85fd-476d1e712cd5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:06.558266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:06.558917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:06.559581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:06.559581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:06.560274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=c037029a4d494bcab2968cdb7f9a9917) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:06.560637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:06.562184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:06.569318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 50/95] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:06 2022] GET /identity/v3/role_assignments?user.id=c037029a4d494bcab2968cdb7f9a9917&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:06.570493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:06.574478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7e7a8-25ec-4eec-b253-ba629a0b5f23 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:06.622008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacb4e2-21fb-490c-a8e0-c0067c6a5634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:06.623064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacb4e2-21fb-490c-a8e0-c0067c6a5634 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:06.623548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacb4e2-21fb-490c-a8e0-c0067c6a5634 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:06.624067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacb4e2-21fb-490c-a8e0-c0067c6a5634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:06.625100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacb4e2-21fb-490c-a8e0-c0067c6a5634 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=272aa82f64454c7aaff95459c4c29980) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:06.626685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacb4e2-21fb-490c-a8e0-c0067c6a5634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:06.628558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacb4e2-21fb-490c-a8e0-c0067c6a5634 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:06.641747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 46/96] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:06 2022] GET /identity/v3/role_assignments?user.id=272aa82f64454c7aaff95459c4c29980&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:06.740221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fd18c-9ec7-4d39-aaa3-28b555839739 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:06.782667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bf965-a10b-4505-9af8-888c5e70c035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:06.783663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf965-a10b-4505-9af8-888c5e70c035 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:06.784158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf965-a10b-4505-9af8-888c5e70c035 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:06.784630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bf965-a10b-4505-9af8-888c5e70c035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:06.785651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf965-a10b-4505-9af8-888c5e70c035 None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=1807b59758d54d56a02fa69faa896ef0, user.id=c037029a4d494bcab2968cdb7f9a9917) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:06.786195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf965-a10b-4505-9af8-888c5e70c035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:06.788100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bf965-a10b-4505-9af8-888c5e70c035 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:06.791863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:06.799989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 51/97] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:06 2022] GET /identity/v3/role_assignments?user.id=c037029a4d494bcab2968cdb7f9a9917&scope.project.id=1807b59758d54d56a02fa69faa896ef0&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:07.875248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486451e7-106b-4128-a320-3250f1043bac None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:07.875885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486451e7-106b-4128-a320-3250f1043bac None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:07.875885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486451e7-106b-4128-a320-3250f1043bac None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:07.877169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 47/98] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:07.883437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628242c-6e61-4fc9-bb23-7f3e1b62b895 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:07.884285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628242c-6e61-4fc9-bb23-7f3e1b62b895 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:07.884793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628242c-6e61-4fc9-bb23-7f3e1b62b895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:07.908285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7628242c-6e61-4fc9-bb23-7f3e1b62b895 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:07.909884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7628242c-6e61-4fc9-bb23-7f3e1b62b895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:08.003006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 52/99] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:08.264435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe2d2d-e39d-4dda-90b4-5b62580409fe None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:08.265106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe2d2d-e39d-4dda-90b4-5b62580409fe None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:08.265600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe2d2d-e39d-4dda-90b4-5b62580409fe None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:08.267142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 48/100] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:08.273333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651b99-3197-40f8-b76c-2b7af1bcd38b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:08.273878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651b99-3197-40f8-b76c-2b7af1bcd38b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:08.274344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0651b99-3197-40f8-b76c-2b7af1bcd38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:08.308047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b0651b99-3197-40f8-b76c-2b7af1bcd38b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:08.309575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b0651b99-3197-40f8-b76c-2b7af1bcd38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:08.401997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 53/101] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:08.512217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe2d2d-e39d-4dda-90b4-5b62580409fe None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:08.555168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:08.646016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe2d2d-e39d-4dda-90b4-5b62580409fe None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:08.693917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:08.694551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:08.694832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:08.695106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:08.696243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:08.697727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:08.699285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 49/102] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:08.705344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651b99-3197-40f8-b76c-2b7af1bcd38b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:08.807604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:08.839608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:08.866245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0651b99-3197-40f8-b76c-2b7af1bcd38b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:08.931935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:08.932546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:08.932855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:08.933132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:08.933799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:08.935664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:08.988977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 54/103] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:45:08 2022] POST /identity/v3/projects => generated 302 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:09.046431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:09.055216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f7eb2-77e6-446f-a6ab-c16f15ca3f2c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:09.102920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:09.103563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.103840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.104109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.116026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:09.121455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:09.122307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:09.123515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 50/104] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:09.128621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:09.263992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae34c-734b-4d89-ba73-eacd7f920c9e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:09.281572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:09.308746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:09.308746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.308746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.308746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.309127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:09.309127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:09.309127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:09.319091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 55/105] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:09.323915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:09.456783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a1d38-8413-439e-951b-1f64f74a1ecf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:09.496445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e3710-d95e-4887-8eb8-9a895139fa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:09.497621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e3710-d95e-4887-8eb8-9a895139fa95 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.498101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e3710-d95e-4887-8eb8-9a895139fa95 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.498592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e3710-d95e-4887-8eb8-9a895139fa95 None admin] PATH_INFO: `/v3/users/526c54ba751c45a6b5b79de56b37f4bc` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.525954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e3710-d95e-4887-8eb8-9a895139fa95 None admin] RBAC: Authorizing `identity:get_user(user_id=526c54ba751c45a6b5b79de56b37f4bc)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:09.527861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e3710-d95e-4887-8eb8-9a895139fa95 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:09.529935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 51/106] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:09 2022] GET /identity/v3/users/526c54ba751c45a6b5b79de56b37f4bc => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:09.535896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:09.666033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9266d-e19d-4508-9db4-6e88e24bd006 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.666662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9266d-e19d-4508-9db4-6e88e24bd006 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.667133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9266d-e19d-4508-9db4-6e88e24bd006 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.668184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ce614-6e54-498b-be52-8b747b6554c7 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:09.669026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 52/107] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:09.682121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62820-e553-447d-9d43-53fe65abc3f8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.682699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62820-e553-447d-9d43-53fe65abc3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.683169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b62820-e553-447d-9d43-53fe65abc3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.729881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c7b62820-e553-447d-9d43-53fe65abc3f8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:09.731034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c7b62820-e553-447d-9d43-53fe65abc3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:09.743038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:09.745039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:09.745709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.745984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.746522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.829132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:09.862484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:09.863437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 53/108] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:09.864399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:09.865366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4b9c6-fd5c-4b01-aa2b-f4f8dd56cb6e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.865652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4b9c6-fd5c-4b01-aa2b-f4f8dd56cb6e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.865924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4b9c6-fd5c-4b01-aa2b-f4f8dd56cb6e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.867252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 54/109] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:09.874742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410955f-d7f9-4827-8d21-744a69a5f07c None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:09.875237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410955f-d7f9-4827-8d21-744a69a5f07c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:09.875642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410955f-d7f9-4827-8d21-744a69a5f07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:09.919866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-4410955f-d7f9-4827-8d21-744a69a5f07c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:09.921335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-4410955f-d7f9-4827-8d21-744a69a5f07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:09.979533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 56/110] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:09.988004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:10.082042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 55/111] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:10.084650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:10.107678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5520-6492-498d-a8b2-9d7b7372167a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:10.144291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:10.145229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.145647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.146229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.146925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059effd2-0189-4aa0-b762-47fdba413abc None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.147233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059effd2-0189-4aa0-b762-47fdba413abc None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.147478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059effd2-0189-4aa0-b762-47fdba413abc None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.147989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:10.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 56/112] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:10.153003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:10.153003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 57/113] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:09 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:10.155488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9216-b793-41f5-a890-7c18a919d087 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.155969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9216-b793-41f5-a890-7c18a919d087 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.156431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9216-b793-41f5-a890-7c18a919d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.166097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:10.199784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-8e1c9216-b793-41f5-a890-7c18a919d087 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:10.201633 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8e1c9216-b793-41f5-a890-7c18a919d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:10.316988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd677e7a-8fc0-4d76-b086-6f9485bd274b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:10.351470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 57/114] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:10.352164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9216-b793-41f5-a890-7c18a919d087 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:10.369418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:10.394864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:10.395507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.395788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.396189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.405246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] RBAC: Adding query filter params (role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=526c54ba751c45a6b5b79de56b37f4bc) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:10.405600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:10.407193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:10.419320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 58/115] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:10 2022] GET /identity/v3/role_assignments?user.id=526c54ba751c45a6b5b79de56b37f4bc&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:10.475187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:10.493194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:10.581034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9216-b793-41f5-a890-7c18a919d087 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:10.633314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebac03-7b43-49ca-8a0e-c375b2455e93 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:10.647357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c065ee-fff2-42bd-b433-1c4c8f313617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:10.648100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c065ee-fff2-42bd-b433-1c4c8f313617 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.648386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c065ee-fff2-42bd-b433-1c4c8f313617 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.648788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c065ee-fff2-42bd-b433-1c4c8f313617 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.659980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c065ee-fff2-42bd-b433-1c4c8f313617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-79c065ee-fff2-42bd-b433-1c4c8f313617 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:10.665414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:10.666261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:10.667028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 58/116] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:10.667846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ee6af-30b1-4979-84d7-4911f6481032 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.668278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ee6af-30b1-4979-84d7-4911f6481032 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.668555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ee6af-30b1-4979-84d7-4911f6481032 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.671149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 59/117] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:10.678746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ee6af-30b1-4979-84d7-4911f6481032 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:10.680104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797fbfb-1110-479f-9d7b-e414410daef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:10.680810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797fbfb-1110-479f-9d7b-e414410daef1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.681091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797fbfb-1110-479f-9d7b-e414410daef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.681362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797fbfb-1110-479f-9d7b-e414410daef1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.701208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797fbfb-1110-479f-9d7b-e414410daef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-0797fbfb-1110-479f-9d7b-e414410daef1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:10.717542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:10.718509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:10.719250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 59/118] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:10.720116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93ed17-38d5-4b6f-b5a0-cfaf86d98ca5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.720538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93ed17-38d5-4b6f-b5a0-cfaf86d98ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.721030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93ed17-38d5-4b6f-b5a0-cfaf86d98ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.753314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6b93ed17-38d5-4b6f-b5a0-cfaf86d98ca5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:10.754949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6b93ed17-38d5-4b6f-b5a0-cfaf86d98ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:10.759748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:10.798504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ee6af-30b1-4979-84d7-4911f6481032 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:10.829714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:10.838098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 60/119] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:10.845748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93ed17-38d5-4b6f-b5a0-cfaf86d98ca5 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:10.851152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:10.851764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:10.852036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:10.852302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:10.853001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:10.853558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:10.855919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:10.875332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 60/120] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:10.876189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.023432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9206a-039a-4626-a925-bc594293e607 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.030752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93ed17-38d5-4b6f-b5a0-cfaf86d98ca5 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.052634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:11.067490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.069110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.070457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.070871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.087637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.088658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.089094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.090301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.091142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:11.091638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.095309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.098413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:11.106902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:11.107899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:11.108674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 61/121] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:11.109344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.117095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 61/122] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.118165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.190895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:11.242330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2ad1-a659-48cd-84ab-02674cfe795b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.243360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc42eee-5b54-4c03-b11a-e6cd638d65e5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.281540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.282481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.283032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.283441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.284363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:11.284471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.284925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.285352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.285726 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.286093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.287608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.288155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.290453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.292469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 62/123] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.293496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.297894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 62/124] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.298948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.384096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:11.439555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803e06b-9788-4d6c-a215-addce8074d79 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.441317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9ec8f-b028-4166-af75-055473681110 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.471611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:11.486985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.487754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.487959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.488166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.489331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.491390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.493022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 63/125] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.497510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.499151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.499811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.500075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.500349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.513096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:11.518449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:11.519314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:11.520074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 63/126] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:11.520742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.640525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f477-00d7-435b-90e0-26e983f0ec87 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.640688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddf8da-0df2-45e5-8e58-19022fb0d1cb None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.665628 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:11.678211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.679268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.679555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.679829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.681126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.682723 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.684489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 64/127] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.685232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.690779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.691678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.692172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.692634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] PATH_INFO: `/v3/users/272aa82f64454c7aaff95459c4c29980` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.694359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] RBAC: Authorizing `identity:get_user(user_id=272aa82f64454c7aaff95459c4c29980)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.696621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.700262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 64/128] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/users/272aa82f64454c7aaff95459c4c29980 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.701684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.773801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:11.805787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c364787-9931-4bcc-b67b-126954323b05 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.835974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7e8c-bfa9-4cdb-84cc-87f2e3295dfb None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:11.851995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.852637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.852944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.853222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.854688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.856894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.858542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 65/129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.863421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.878955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:11.879577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:11.879857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:11.880129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:11.881545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:11.882288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:11.884089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:11.899508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 65/130] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:11.899508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:11.937087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:11.987333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47ee6a-88bd-4f5f-93b0-8adf8ab1893e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:12.008206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296e878-a72a-4d87-8870-af7e7416d731 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:12.018789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:12.026520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:12.027546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.027996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.028532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/c037029a4d494bcab2968cdb7f9a9917/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.030659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7ac2137e95a4380a28ffb6603712f74, user_id=c037029a4d494bcab2968cdb7f9a9917, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:12.032403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:12.046513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:12.047151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.047563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.047902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.049275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:12.050888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 66/131] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:11 2022] PUT /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/c037029a4d494bcab2968cdb7f9a9917/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:12.051199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:12.052710 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:12.054180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 66/132] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:11 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:12.059222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:12.175007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ce045-cbee-4c1a-805a-2cd7f5d9ccfc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:12.202401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12599fe-f0f3-41ed-868d-88a1e001e813 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:12.214898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:12.215565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.215844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.216412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.217498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:12.219083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:12.226140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 67/133] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:12 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:12.227230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:12.229328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:12.248750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c6de5-d727-4f18-b436-9197c8297465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:12.249961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6de5-d727-4f18-b436-9197c8297465 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.249961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6de5-d727-4f18-b436-9197c8297465 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.250139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6de5-d727-4f18-b436-9197c8297465 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/c037029a4d494bcab2968cdb7f9a9917/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.254129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c6de5-d727-4f18-b436-9197c8297465 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7ac2137e95a4380a28ffb6603712f74, user_id=c037029a4d494bcab2968cdb7f9a9917, role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:12.254820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c6de5-d727-4f18-b436-9197c8297465 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:12.276533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 67/134] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:12 2022] PUT /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/c037029a4d494bcab2968cdb7f9a9917/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:12.277623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a22d4a-f318-4dc8-94eb-3c8e0a7fcc84 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.277915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a22d4a-f318-4dc8-94eb-3c8e0a7fcc84 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.278191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a22d4a-f318-4dc8-94eb-3c8e0a7fcc84 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.279427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 68/135] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:12.280134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a22d4a-f318-4dc8-94eb-3c8e0a7fcc84 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:12.296342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:12.378553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce156a-5415-4ef5-bfff-1a9d4c45bed6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:12.404231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a22d4a-f318-4dc8-94eb-3c8e0a7fcc84 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:12.430861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9141b9-8937-44e1-ab42-c84e1ba3c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:12.431480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9141b9-8937-44e1-ab42-c84e1ba3c35a None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.431762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9141b9-8937-44e1-ab42-c84e1ba3c35a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.432035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9141b9-8937-44e1-ab42-c84e1ba3c35a None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/272aa82f64454c7aaff95459c4c29980/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.438703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9141b9-8937-44e1-ab42-c84e1ba3c35a None admin] RBAC: Authorizing `identity:create_grant(project_id=d7ac2137e95a4380a28ffb6603712f74, user_id=272aa82f64454c7aaff95459c4c29980, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:12.443645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9141b9-8937-44e1-ab42-c84e1ba3c35a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:12.465532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:12.467632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.467924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.468201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] PATH_INFO: `/v3/projects/1807b59758d54d56a02fa69faa896ef0` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.471398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 68/136] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:12 2022] PUT /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/272aa82f64454c7aaff95459c4c29980/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:12.472689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ceea6-d124-4975-9d09-d9be144eb9ef None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.473006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ceea6-d124-4975-9d09-d9be144eb9ef None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.473558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ceea6-d124-4975-9d09-d9be144eb9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.475366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] RBAC: Authorizing `identity:get_project(project_id=1807b59758d54d56a02fa69faa896ef0)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:12.477015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:12.481752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 69/137] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:12 2022] GET /identity/v3/projects/1807b59758d54d56a02fa69faa896ef0 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:12.487581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:12.510256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-8b4ceea6-d124-4975-9d09-d9be144eb9ef None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:12.511473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8b4ceea6-d124-4975-9d09-d9be144eb9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:12.564558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:12.646965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 69/138] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:12.660385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123734d-4c74-4c27-8d70-0d2ec40fd1ff None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:12.697906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9848a6-4edf-4deb-9cb6-a08444cdcd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:12.698501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9848a6-4edf-4deb-9cb6-a08444cdcd8c None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:12.698779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9848a6-4edf-4deb-9cb6-a08444cdcd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:12.699046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9848a6-4edf-4deb-9cb6-a08444cdcd8c None admin] PATH_INFO: `/v3/projects/1807b59758d54d56a02fa69faa896ef0/users/c037029a4d494bcab2968cdb7f9a9917/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:12.701020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9848a6-4edf-4deb-9cb6-a08444cdcd8c None admin] RBAC: Authorizing `identity:create_grant(project_id=1807b59758d54d56a02fa69faa896ef0, user_id=c037029a4d494bcab2968cdb7f9a9917, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:12.702500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9848a6-4edf-4deb-9cb6-a08444cdcd8c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:12.725207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 70/139] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:12 2022] PUT /identity/v3/projects/1807b59758d54d56a02fa69faa896ef0/users/c037029a4d494bcab2968cdb7f9a9917/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:13.031062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed07d-9ce9-41e0-ac71-1cb867da6cd3 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:13.031407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed07d-9ce9-41e0-ac71-1cb867da6cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:13.031577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed07d-9ce9-41e0-ac71-1cb867da6cd3 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:13.033147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 70/140] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:13.038338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ae5c6-595a-4ec2-a324-90b67155dd4b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:13.038621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ae5c6-595a-4ec2-a324-90b67155dd4b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:13.038895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ae5c6-595a-4ec2-a324-90b67155dd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:13.049228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed07d-9ce9-41e0-ac71-1cb867da6cd3 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:13.062811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-585ae5c6-595a-4ec2-a324-90b67155dd4b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:13.064021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-585ae5c6-595a-4ec2-a324-90b67155dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:13.092327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:13.156955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 71/141] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:13.209566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed07d-9ce9-41e0-ac71-1cb867da6cd3 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:13.265625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:13.266278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:13.266561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:13.266835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:13.267980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:13.269475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:13.271365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 71/142] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:13.277128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ae5c6-595a-4ec2-a324-90b67155dd4b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:13.351347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:13.426572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ae5c6-595a-4ec2-a324-90b67155dd4b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:13.477499 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:13.478143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:13.478429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:13.478703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:13.479377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:13.480915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:13.488270 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:13.522811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 72/143] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:45:13 2022] POST /identity/v3/users => generated 283 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:13.645271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ccd1a-a040-4cea-b4da-868f60f3030d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:13.648527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:13.703686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:13.704342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:13.704617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:13.704908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:13.722934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:13.729125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:13.729984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:13.730805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:13.737821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:13.883133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:13.890192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3113-6c84-4c1b-b7e4-243fd311efd5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:13.940972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:13.941508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:13.941785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:13.942059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:13.943347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:13.943847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:13.945296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:13.959195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 73/145] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:13.964981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:14.103651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19753f7-1cc9-4468-bfb9-2e4aa1eba795 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:14.158604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:14.159653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:14.160146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:14.160640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] PATH_INFO: `/v3/users/526c54ba751c45a6b5b79de56b37f4bc` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:14.162811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] RBAC: Authorizing `identity:get_user(user_id=526c54ba751c45a6b5b79de56b37f4bc)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:14.164506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:14.166468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:13 2022] GET /identity/v3/users/526c54ba751c45a6b5b79de56b37f4bc => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:14.172301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:14.298318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d503f9-9c1e-4c60-95ce-7f2cf2d50ec3 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:14.348780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a949c7-9901-4154-9a29-e8ba61da5c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:14.349753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a949c7-9901-4154-9a29-e8ba61da5c2e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:14.350097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a949c7-9901-4154-9a29-e8ba61da5c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:14.350375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a949c7-9901-4154-9a29-e8ba61da5c2e None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:14.351533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a949c7-9901-4154-9a29-e8ba61da5c2e None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:14.353024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a949c7-9901-4154-9a29-e8ba61da5c2e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:14.355068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 74/147] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:14 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:14.360867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:14.398944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:14.482744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022020d-058f-4c42-bfba-422b34b8857c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:14.527497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42723e65-cf00-4f19-b57e-1e3c1d5c15c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:14.528172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723e65-cf00-4f19-b57e-1e3c1d5c15c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:14.528462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723e65-cf00-4f19-b57e-1e3c1d5c15c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:14.528750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723e65-cf00-4f19-b57e-1e3c1d5c15c9 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/526c54ba751c45a6b5b79de56b37f4bc/roles/5182eb7e9f3a4b9b8d563e8bf9888d0f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:14.533892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42723e65-cf00-4f19-b57e-1e3c1d5c15c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7ac2137e95a4380a28ffb6603712f74, user_id=526c54ba751c45a6b5b79de56b37f4bc, role_id=5182eb7e9f3a4b9b8d563e8bf9888d0f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:14.535366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42723e65-cf00-4f19-b57e-1e3c1d5c15c9 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:14.559223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 74/148] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:14 2022] PUT /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74/users/526c54ba751c45a6b5b79de56b37f4bc/roles/5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:15.343747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8d55f-8613-4b62-bd3c-149fdae8da28 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:15.344259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8d55f-8613-4b62-bd3c-149fdae8da28 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:15.344349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8d55f-8613-4b62-bd3c-149fdae8da28 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:15.345677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 75/149] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:15.351424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d295a58-ab48-4650-9128-19708db97a10 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:15.351868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d295a58-ab48-4650-9128-19708db97a10 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:15.352066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d295a58-ab48-4650-9128-19708db97a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:15.374794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6d295a58-ab48-4650-9128-19708db97a10 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:15.376003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6d295a58-ab48-4650-9128-19708db97a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:15.441716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a2d1a-2de3-49a6-9cf6-38abd0fcb23a None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:15.442036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a2d1a-2de3-49a6-9cf6-38abd0fcb23a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:15.442277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a2d1a-2de3-49a6-9cf6-38abd0fcb23a None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:15.444184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 76/150] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:15.451561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e46d-b431-41cf-9849-5f64c2262664 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:15.451846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e46d-b431-41cf-9849-5f64c2262664 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:15.452124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519e46d-b431-41cf-9849-5f64c2262664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:15.458620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 75/151] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:15.485622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2519e46d-b431-41cf-9849-5f64c2262664 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:15.486833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2519e46d-b431-41cf-9849-5f64c2262664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:15.532839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:15.597308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 77/152] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:16.031284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d295a58-ab48-4650-9128-19708db97a10 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:16.060672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:16.111554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c93f8-cd12-478c-b52f-55e89b0518b1 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.111872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c93f8-cd12-478c-b52f-55e89b0518b1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.112104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c93f8-cd12-478c-b52f-55e89b0518b1 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.113618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 78/153] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:16.119381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018acb13-73c3-4817-b514-448cfb2cf9bd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.119656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018acb13-73c3-4817-b514-448cfb2cf9bd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.119923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018acb13-73c3-4817-b514-448cfb2cf9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.150486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-018acb13-73c3-4817-b514-448cfb2cf9bd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:16.151690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-018acb13-73c3-4817-b514-448cfb2cf9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:16.206929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d295a58-ab48-4650-9128-19708db97a10 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:16.260024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 79/154] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:16.260923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018acb13-73c3-4817-b514-448cfb2cf9bd None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:16.317220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:16.318416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.318686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.318952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.361975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:16.367298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:16.368250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:16.369046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 76/155] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:16.382358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:16.421763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:16.453229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018acb13-73c3-4817-b514-448cfb2cf9bd None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:16.505871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36a6ac-a5ad-4d66-ae67-f7d8715ff691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:16.506991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36a6ac-a5ad-4d66-ae67-f7d8715ff691 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.508694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36a6ac-a5ad-4d66-ae67-f7d8715ff691 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.509220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36a6ac-a5ad-4d66-ae67-f7d8715ff691 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.531217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36a6ac-a5ad-4d66-ae67-f7d8715ff691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-ae36a6ac-a5ad-4d66-ae67-f7d8715ff691 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:16.538690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:16.539543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:16.540523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:16.547230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 80/156] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:16.557634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afc968-450d-460f-9f34-eea8c1e04b6d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.558243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afc968-450d-460f-9f34-eea8c1e04b6d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.558718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afc968-450d-460f-9f34-eea8c1e04b6d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.560257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 81/157] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:16.564134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:16.564353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afc968-450d-460f-9f34-eea8c1e04b6d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:16.602730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391656e-b06c-47da-8d59-e1a4c79cf0d1 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:16.644351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54d0d4-d965-4637-92eb-1dd5eff1d4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:16.647039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d0d4-d965-4637-92eb-1dd5eff1d4dc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.647557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d0d4-d965-4637-92eb-1dd5eff1d4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.648239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d0d4-d965-4637-92eb-1dd5eff1d4dc None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.649817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54d0d4-d965-4637-92eb-1dd5eff1d4dc None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:16.650500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54d0d4-d965-4637-92eb-1dd5eff1d4dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:16.653768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54d0d4-d965-4637-92eb-1dd5eff1d4dc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:16.680430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 77/158] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:16.682194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c49e6-28a0-40f9-93d2-cd115bd0cc2f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.685940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c49e6-28a0-40f9-93d2-cd115bd0cc2f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.686467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c49e6-28a0-40f9-93d2-cd115bd0cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.733174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0c7c49e6-28a0-40f9-93d2-cd115bd0cc2f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:16.735216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0c7c49e6-28a0-40f9-93d2-cd115bd0cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:16.778734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afc968-450d-460f-9f34-eea8c1e04b6d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:16.845406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:16.846249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:16.846532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:16.846798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:16.847475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:16.847774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:16.849902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:16.864161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 82/159] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:16.865639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:16.880129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:16.892965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 78/160] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:16.894612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c49e6-28a0-40f9-93d2-cd115bd0cc2f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:17.025390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:17.037300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd97003-15eb-41c1-b114-006a26d9a5ca None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:17.075112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c49e6-28a0-40f9-93d2-cd115bd0cc2f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:17.112308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:17.113018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:17.113299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:17.113585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:17.115378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:17.116884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:17.119837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 83/161] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:16 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:17.120640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:17.138388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:17.139034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:17.139306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:17.139567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:17.145412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:17.147039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:17.148656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 79/162] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:16 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:17.149447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:17.263709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:17.280657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f34b6-5bf0-4027-a29a-66f2a2395cce None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:17.343055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:17.343698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:17.343976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:17.344250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:17.361370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:17.367543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:17.368400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:17.369209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 84/163] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:17.373578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:17.430888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32857-fa5f-447f-892a-fa64e862f71c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:17.526417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:17.527040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:17.527315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:17.527588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:17.529747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4419b30-4ee2-4eeb-acf6-96aa69afaaae None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:17.552586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:17.594518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:17.595151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:17.595431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:17.595704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:17.603225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:17.725628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:17.727032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:17.728245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:17.730715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:17.924527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 85/164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:17.925543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:17.997167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:18.000663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 80/165] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 851 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:18.001596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.070570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:18.129677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d282cfd-194b-47d4-b105-3a2799a33c0a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.169526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727e48-80e5-4c04-943d-1123001a1135 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.227583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.229112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.230226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.231599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.233061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:18.233911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.234443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:18.235297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.236985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:18.237462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.237924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.253195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 86/166] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:18.255657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.260363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:18.265944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:18.266792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:18.267651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:18.269893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 81/167] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:18.271510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.400003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:18.406349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c9398-2a31-4520-aa0f-eb852dc90478 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.424484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9d56-aa1e-481c-8d79-2028e5fc630b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.426664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:18.446351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c476e-bab0-45bc-ae59-5013ae0bdbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.447304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c476e-bab0-45bc-ae59-5013ae0bdbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.447758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c476e-bab0-45bc-ae59-5013ae0bdbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.448103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c476e-bab0-45bc-ae59-5013ae0bdbf8 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.449523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c476e-bab0-45bc-ae59-5013ae0bdbf8 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:18.451375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c476e-bab0-45bc-ae59-5013ae0bdbf8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:18.453290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 87/168] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:18.457763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c90232-a576-454c-b94d-bbbd04ba018e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.458455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c90232-a576-454c-b94d-bbbd04ba018e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.458897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c90232-a576-454c-b94d-bbbd04ba018e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.461390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 88/169] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:18.462405 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c90232-a576-454c-b94d-bbbd04ba018e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.475611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.476608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.476608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.476977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.478205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:18.479735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:18.481661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 82/170] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:18.482371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.579791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c90232-a576-454c-b94d-bbbd04ba018e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.591717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eb148-ae0e-4099-8fb0-43a6246ddffe None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.622825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099799c-d03a-4543-a956-85f3bbb502d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.623752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099799c-d03a-4543-a956-85f3bbb502d0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.624267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099799c-d03a-4543-a956-85f3bbb502d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.624784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099799c-d03a-4543-a956-85f3bbb502d0 None admin] PATH_INFO: `/v3/users/c037029a4d494bcab2968cdb7f9a9917` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.626280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099799c-d03a-4543-a956-85f3bbb502d0 None admin] RBAC: Authorizing `identity:get_user(user_id=c037029a4d494bcab2968cdb7f9a9917)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:18.628204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099799c-d03a-4543-a956-85f3bbb502d0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:18.629920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01473da5-6b9a-417f-9efd-42fbdebeade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.630873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 89/171] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/users/c037029a4d494bcab2968cdb7f9a9917 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:18.631188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01473da5-6b9a-417f-9efd-42fbdebeade7 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.631600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01473da5-6b9a-417f-9efd-42fbdebeade7 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.631944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01473da5-6b9a-417f-9efd-42fbdebeade7 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.632683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01473da5-6b9a-417f-9efd-42fbdebeade7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:18.633030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b1671-22ec-4552-956d-ec70a8f96572 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.633146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01473da5-6b9a-417f-9efd-42fbdebeade7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:18.633596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b1671-22ec-4552-956d-ec70a8f96572 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.634039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b1671-22ec-4552-956d-ec70a8f96572 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.634681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01473da5-6b9a-417f-9efd-42fbdebeade7 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:18.635757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 90/172] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:18.636827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b1671-22ec-4552-956d-ec70a8f96572 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.648468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 83/173] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:18.653156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452f06-13a5-4bcb-9d70-231d38ece9e8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.653156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452f06-13a5-4bcb-9d70-231d38ece9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.653156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3452f06-13a5-4bcb-9d70-231d38ece9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.669510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b3452f06-13a5-4bcb-9d70-231d38ece9e8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:18.670797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b3452f06-13a5-4bcb-9d70-231d38ece9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:18.678189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:18.700427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:18.748220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 84/174] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:18.749122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3452f06-13a5-4bcb-9d70-231d38ece9e8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.766919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b1671-22ec-4552-956d-ec70a8f96572 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.817249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.818095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.818378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.818652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.819397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] RBAC: Adding query filter params (role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=c037029a4d494bcab2968cdb7f9a9917) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:18.819708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:18.822077 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:18.830075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 91/175] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/role_assignments?user.id=c037029a4d494bcab2968cdb7f9a9917&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:18.834396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:18.892421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3452f06-13a5-4bcb-9d70-231d38ece9e8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:18.965803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a1191-d0bf-4264-9857-11819a4f2ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:18.966847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a1191-d0bf-4264-9857-11819a4f2ebb None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.967318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a1191-d0bf-4264-9857-11819a4f2ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.967786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a1191-d0bf-4264-9857-11819a4f2ebb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:18.968761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a1191-d0bf-4264-9857-11819a4f2ebb None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=c037029a4d494bcab2968cdb7f9a9917) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:18.969280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a1191-d0bf-4264-9857-11819a4f2ebb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:18.970949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a1191-d0bf-4264-9857-11819a4f2ebb None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:18.978242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 85/176] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/role_assignments?user.id=c037029a4d494bcab2968cdb7f9a9917&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:18.980045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2450a40-374c-4eee-8165-d3ea983e5b86 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:18.980588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2450a40-374c-4eee-8165-d3ea983e5b86 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:18.981073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2450a40-374c-4eee-8165-d3ea983e5b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.013803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e2450a40-374c-4eee-8165-d3ea983e5b86 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:19.018476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e2450a40-374c-4eee-8165-d3ea983e5b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:19.038193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.046605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b2469-4ec4-45f9-9b0d-a03f255296f7 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:19.049057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.094707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:19.095226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.095498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.095764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.117059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 86/177] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:19.118190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2450a40-374c-4eee-8165-d3ea983e5b86 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.201124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:19.202639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:19.263262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2450a40-374c-4eee-8165-d3ea983e5b86 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:19.311303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a58ddf-2c16-46b5-acdf-65e1f61b8052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:19.312105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58ddf-2c16-46b5-acdf-65e1f61b8052 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.315202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58ddf-2c16-46b5-acdf-65e1f61b8052 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.315202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a58ddf-2c16-46b5-acdf-65e1f61b8052 None admin] PATH_INFO: `/v3/users/272aa82f64454c7aaff95459c4c29980` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.318542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a58ddf-2c16-46b5-acdf-65e1f61b8052 None admin] RBAC: Authorizing `identity:get_user(user_id=272aa82f64454c7aaff95459c4c29980)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:19.319515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a58ddf-2c16-46b5-acdf-65e1f61b8052 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:19.321584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 87/178] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/users/272aa82f64454c7aaff95459c4c29980 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:19.322931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657225f-93ad-4c32-a89f-909d3819c872 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.323215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657225f-93ad-4c32-a89f-909d3819c872 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.323496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657225f-93ad-4c32-a89f-909d3819c872 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.328747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 88/179] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:19.329189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.330191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaafaf5-e960-4188-9e67-b42e0c4b1fe5 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.330480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaafaf5-e960-4188-9e67-b42e0c4b1fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.330782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaafaf5-e960-4188-9e67-b42e0c4b1fe5 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.332042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 89/180] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:19.334230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaafaf5-e960-4188-9e67-b42e0c4b1fe5 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.338296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 92/181] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:19.338975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.360692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.448227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024732ad-eb3d-4435-81ee-80a07b0fdcb1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:19.450479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaafaf5-e960-4188-9e67-b42e0c4b1fe5 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:19.483219 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:19.483825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.484101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.484368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.485101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7bb97-21a5-4c68-a66f-b2eb342bee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:19.485728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7bb97-21a5-4c68-a66f-b2eb342bee1d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.485998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7bb97-21a5-4c68-a66f-b2eb342bee1d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.486259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7bb97-21a5-4c68-a66f-b2eb342bee1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.496701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7bb97-21a5-4c68-a66f-b2eb342bee1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-9ed7bb97-21a5-4c68-a66f-b2eb342bee1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:19.501417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:19.502488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:19.503321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 90/182] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:19.503960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b611c7b-d901-48e2-848e-671de0dc6f46 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.503960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b611c7b-d901-48e2-848e-671de0dc6f46 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.504123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b611c7b-d901-48e2-848e-671de0dc6f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.522108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7b611c7b-d901-48e2-848e-671de0dc6f46 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:19.523298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7b611c7b-d901-48e2-848e-671de0dc6f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:19.547728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.555448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:19.556960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:19.590916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 91/183] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:19.592158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96fc4-a1be-4908-a573-8e6de566f36e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.592436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96fc4-a1be-4908-a573-8e6de566f36e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.592703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a96fc4-a1be-4908-a573-8e6de566f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.611401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-35a96fc4-a1be-4908-a573-8e6de566f36e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:19.612521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-35a96fc4-a1be-4908-a573-8e6de566f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:19.624331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.683424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 93/184] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:19.683424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.691043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 92/185] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:19.693272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96fc4-a1be-4908-a573-8e6de566f36e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.782293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.788140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c0589-83da-434d-9095-81e513b66b9f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:19.815073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a96fc4-a1be-4908-a573-8e6de566f36e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:19.824037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:19.824814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.825274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.825639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] PATH_INFO: `/v3/projects/1807b59758d54d56a02fa69faa896ef0` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.826923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] RBAC: Authorizing `identity:get_project(project_id=1807b59758d54d56a02fa69faa896ef0)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:19.828478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:19.830270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 94/186] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/projects/1807b59758d54d56a02fa69faa896ef0 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:19.831137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.855175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:19.856137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.856582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.857024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.869142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:19.874199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:19.875085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:19.876626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 93/187] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:19.881144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.885302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:19.939584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3db7-4614-4e05-91eb-e54ad5dff8fc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:19.974938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:19.975543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:19.975817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:19.976081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:19.976757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:19.977061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:19.978463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:19.986383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 95/188] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:19.987176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:19.996545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0115973-81d2-49e5-9912-ab4278ef2244 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.014182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.029560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.030181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.030448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.030715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.031823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.033305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.034930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 94/189] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.035666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.089225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf1fa7-c4ec-43f5-8223-ad98be2f2c7a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.108346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.123902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.124435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.124713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.125014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.125745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=1807b59758d54d56a02fa69faa896ef0, user.id=c037029a4d494bcab2968cdb7f9a9917) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:20.126052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.127513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.133469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 96/190] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:19 2022] GET /identity/v3/role_assignments?user.id=c037029a4d494bcab2968cdb7f9a9917&scope.project.id=1807b59758d54d56a02fa69faa896ef0&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.134263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.138075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de5169-9bd7-4877-8ff6-bb4b54dd9754 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.180085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.180863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.181151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.181428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.195221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:20.201016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:20.201971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:20.202756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 95/191] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:20.206480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.254360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.265220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913457bb-3e32-4431-a98a-2fc09928a9ae None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.308816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.309454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.310287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.310287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.310828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:20.311432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.312904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.326249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 97/192] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.327097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.332653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e13264-770b-4e65-8767-f2cf19eea81e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.366230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.372080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.372560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.372988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.373272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.374462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.376013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.377983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 96/193] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.378832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.443168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c08f9a-f653-41d8-838c-34d1d78255a3 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.476218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e29044-b913-458e-8530-918ba305b97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.476873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e29044-b913-458e-8530-918ba305b97b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.477156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e29044-b913-458e-8530-918ba305b97b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.477428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e29044-b913-458e-8530-918ba305b97b None admin] PATH_INFO: `/v3/users/526c54ba751c45a6b5b79de56b37f4bc` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.478564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e29044-b913-458e-8530-918ba305b97b None admin] RBAC: Authorizing `identity:get_user(user_id=526c54ba751c45a6b5b79de56b37f4bc)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.479557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1504c38-5651-4be8-8f6e-3d3c44002331 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.480030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e29044-b913-458e-8530-918ba305b97b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.481654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 98/194] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/users/526c54ba751c45a6b5b79de56b37f4bc => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.482426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e29044-b913-458e-8530-918ba305b97b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.504093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.518251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.518856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.519129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.519394 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.520111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=272aa82f64454c7aaff95459c4c29980) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:20.520411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.521938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.527711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 97/195] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/role_assignments?user.id=272aa82f64454c7aaff95459c4c29980&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.528736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.596631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e29044-b913-458e-8530-918ba305b97b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.613155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.635152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.635690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.635972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.636245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.636952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:20.637261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.638686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.647911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 99/196] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.653601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.657889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa126a-be06-4b51-a90d-76d5d689475c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.696238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.696933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.697208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.697474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] PATH_INFO: `/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.714126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] RBAC: Authorizing `identity:get_user(user_id=5cec43c4980d4c05815e0e9ad64b6dc2)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.715621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.718034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 98/197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:20.721533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.748699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.772639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466ad9f-66a6-4ec4-9984-1987f41e3fe0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.810123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.810738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.811016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.811289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.811952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.813451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:20.835399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 100/198] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:45:20 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:20.835399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:20.844470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fd5b4-5317-424b-8996-b60833a8f079 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.869751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:20.885864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:20.886666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:20.886948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:20.887220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:20.964832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8a7f6-70dc-4614-8732-1b774d68ad3e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:20.970076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:20.971587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.003484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.004128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.004407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.004679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] PATH_INFO: `/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.006551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] RBAC: Authorizing `identity:get_user(user_id=5cec43c4980d4c05815e0e9ad64b6dc2)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.008305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.009665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 101/199] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:21.010532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.021261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.092646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 99/200] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:21.093551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.120053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.136979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d579-296c-46f6-8f7b-d17bd3fc4a1c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:21.176685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.177896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.178394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.179425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.210889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc01ae-2a27-43e7-ae50-7d37dcbc1684 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:21.249796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.250455 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.250736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.251004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.261373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.262974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.276981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.352127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.354131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.384649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.391042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 102/201] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:21.392157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.480822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 100/202] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:21.482588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.528290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed83a8a-ca0d-4a33-aac2-da1f75772d48 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:21.540619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.571404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.572031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.572318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.572592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.573778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.575647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.577917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 103/203] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:21.579045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.609069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ce685-6423-4d10-8831-0f71ec19610a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:21.636074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.648837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.649418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.649703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.649976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.651301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.653757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.660210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 101/204] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:21.660210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.688535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d840b3-f301-49f5-bc98-6b9f3041315f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:21.728023 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.729040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.729287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.729561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.730958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.732514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.734663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 104/205] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:21.735491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.783186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.787513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd315232-285e-49dc-b785-0b7404f4d8fc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:21.826530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.827501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.828001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.828622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.829943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] RBAC: Adding query filter params (role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f, scope.project.id=d7ac2137e95a4380a28ffb6603712f74, user.id=526c54ba751c45a6b5b79de56b37f4bc) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:21.830712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.832355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.840932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 102/206] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/role_assignments?user.id=526c54ba751c45a6b5b79de56b37f4bc&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:21.842261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:21.850287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7206c8-95c1-42d7-b095-207104c6f1ea None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:21.890088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9f2fc-9b90-407f-8f85-e38a518ba485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:21.890809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f2fc-9b90-407f-8f85-e38a518ba485 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:21.891157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f2fc-9b90-407f-8f85-e38a518ba485 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:21.891497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9f2fc-9b90-407f-8f85-e38a518ba485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:21.892466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9f2fc-9b90-407f-8f85-e38a518ba485 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=5cec43c4980d4c05815e0e9ad64b6dc2) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:21.892873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9f2fc-9b90-407f-8f85-e38a518ba485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:21.894497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f9f2fc-9b90-407f-8f85-e38a518ba485 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:21.906289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 105/207] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/role_assignments?user.id=5cec43c4980d4c05815e0e9ad64b6dc2&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:21.918793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:21.997037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c807a2-da5d-4e6b-9bf5-5edc7d8fecf3 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:22.050181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f00842-c72b-4d04-bd1d-57f29fee6295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:22.050812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f00842-c72b-4d04-bd1d-57f29fee6295 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:22.051084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f00842-c72b-4d04-bd1d-57f29fee6295 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:22.051352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f00842-c72b-4d04-bd1d-57f29fee6295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:22.052628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f00842-c72b-4d04-bd1d-57f29fee6295 None admin] RBAC: Adding query filter params (role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=5cec43c4980d4c05815e0e9ad64b6dc2) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:22.052985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f00842-c72b-4d04-bd1d-57f29fee6295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:22.054405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f00842-c72b-4d04-bd1d-57f29fee6295 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:22.060665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 103/208] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:21 2022] GET /identity/v3/role_assignments?user.id=5cec43c4980d4c05815e0e9ad64b6dc2&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:23.255171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a265ac-cfc6-47ca-bd75-0ad943d4cfc9 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:23.255708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a265ac-cfc6-47ca-bd75-0ad943d4cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:23.256049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a265ac-cfc6-47ca-bd75-0ad943d4cfc9 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:23.257811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 106/209] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:23.264008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175b848-50ca-48e5-bf12-8219e3955efa None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:23.264520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175b848-50ca-48e5-bf12-8219e3955efa None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:23.264592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175b848-50ca-48e5-bf12-8219e3955efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:23.284550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-1175b848-50ca-48e5-bf12-8219e3955efa None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:23.285780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1175b848-50ca-48e5-bf12-8219e3955efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:23.365480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 104/210] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:23.379328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c605461-ed54-4fb6-8c6d-64675e81ea8d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:23.379607 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c605461-ed54-4fb6-8c6d-64675e81ea8d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:23.379967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c605461-ed54-4fb6-8c6d-64675e81ea8d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:23.381980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 107/211] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:23.387294 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83862a55-ca65-4131-bfa6-a67a5e02ed93 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:23.387740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83862a55-ca65-4131-bfa6-a67a5e02ed93 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:23.388170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83862a55-ca65-4131-bfa6-a67a5e02ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:23.413300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-83862a55-ca65-4131-bfa6-a67a5e02ed93 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:23.414498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-83862a55-ca65-4131-bfa6-a67a5e02ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:23.462179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:23.484196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 105/212] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:23.649062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c605461-ed54-4fb6-8c6d-64675e81ea8d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:23.759601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83862a55-ca65-4131-bfa6-a67a5e02ed93 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:23.767643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c605461-ed54-4fb6-8c6d-64675e81ea8d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:23.776470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:23.814923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:23.815554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:23.815839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:23.816108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:23.827969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:23.832775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:23.833786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:23.834715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:23.835307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 108/213] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:23.840470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:23.879488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83862a55-ca65-4131-bfa6-a67a5e02ed93 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:23.921969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:23.922934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:23.923435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:23.924340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:23.925732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:23.927902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:23.935557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 106/214] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:23.936991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:23.952630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdeefc1-2236-4d8f-a4a4-c8af2333cf54 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:23.991236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:23.991854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:23.992131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:23.992443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:23.993159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:23.993514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:23.994974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:23.997583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:24.009071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 109/215] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:24.011242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:24.046923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:24.064496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef09616-2db0-45e1-989f-f769904be808 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:24.103308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90e80d-862f-4f46-a568-77cfb24947c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:24.104276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90e80d-862f-4f46-a568-77cfb24947c0 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.105115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90e80d-862f-4f46-a568-77cfb24947c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.105633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90e80d-862f-4f46-a568-77cfb24947c0 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.106433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90e80d-862f-4f46-a568-77cfb24947c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:24.108745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90e80d-862f-4f46-a568-77cfb24947c0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:24.134299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab6dd5-3542-40ee-bcc3-3b5c8679a65e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:24.141266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 107/216] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:45:23 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:24.150858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1180b-c0e6-4ceb-b64a-8fca24dbb09d None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.151182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1180b-c0e6-4ceb-b64a-8fca24dbb09d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.151404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1180b-c0e6-4ceb-b64a-8fca24dbb09d None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.153413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 108/217] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:24.159015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c2425-3267-4f83-bb29-27d2484a124f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.159327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c2425-3267-4f83-bb29-27d2484a124f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.159555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c2425-3267-4f83-bb29-27d2484a124f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.180246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f07c2425-3267-4f83-bb29-27d2484a124f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:24.181695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f07c2425-3267-4f83-bb29-27d2484a124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:24.185069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:24.185787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.186069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.186365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] PATH_INFO: `/v3/users/db69d9c655df416d82282b0e261e1049` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.203631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] RBAC: Authorizing `identity:get_user(user_id=db69d9c655df416d82282b0e261e1049)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:24.205173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:24.212549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 110/218] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:24 2022] GET /identity/v3/users/db69d9c655df416d82282b0e261e1049 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:24.213760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:24.264886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 109/219] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:24.341906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:24.361383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293636-de8c-4922-951c-6d3a92a7dddf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:24.397314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:24.398096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.398375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.398646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.436289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56975b61-4a6b-4a25-ba29-4141b79f5ca4 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.436563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56975b61-4a6b-4a25-ba29-4141b79f5ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.436793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56975b61-4a6b-4a25-ba29-4141b79f5ca4 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.438297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 110/220] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:24.444029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd978b-14a6-4dfa-96e3-8b4dbff402d7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.444309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd978b-14a6-4dfa-96e3-8b4dbff402d7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.444584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd978b-14a6-4dfa-96e3-8b4dbff402d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.467039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d9bd978b-14a6-4dfa-96e3-8b4dbff402d7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:24.468215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d9bd978b-14a6-4dfa-96e3-8b4dbff402d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:24.478549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:24.480575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:24.504424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:24.563824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 111/221] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:24.564673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd978b-14a6-4dfa-96e3-8b4dbff402d7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:24.584510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:24.610125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 111/222] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:24.616430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:24.692982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd978b-14a6-4dfa-96e3-8b4dbff402d7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:24.747748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb0e86-3264-4c80-97ee-d3bf70a4d0c6 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:24.778184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:24.778838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.779113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.779380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.784757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:24.785554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.785844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.786318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.787745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:24.789326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:24.791582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 112/223] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:24 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:24.797009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:24.817836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:24.822689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:24.823529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:24.824697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 112/224] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:24.834282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:24.865083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:24.889938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:24.931388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058367bb-9573-4af2-a59f-ebdcb4368db5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:24.973328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4d4b4-96bd-4a1c-922d-1c37ffc139a8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:24.983116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:24.983765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:24.983993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:24.984239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:24.985012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=db69d9c655df416d82282b0e261e1049) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:24.985315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:24.986843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:24.992753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 113/225] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:24 2022] GET /identity/v3/role_assignments?user.id=db69d9c655df416d82282b0e261e1049&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:24.993379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:25.019166 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.020247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.021308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.021984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.023444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:25.023984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:25.026046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:25.037709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 113/226] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:25.045880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:25.201256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b6533-5390-4964-9ea5-b39b1fd59626 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:25.219318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:25.228520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:25.250996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc52dfb-0004-4ffa-8f12-4c8867527261 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:25.253704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.254582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.255086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.255548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.268137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:25.275092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:25.275983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:25.276806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:25.279198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 114/227] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:25.285773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:25.349980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.350887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.351162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.351428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] PATH_INFO: `/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.357472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] RBAC: Authorizing `identity:get_user(user_id=5cec43c4980d4c05815e0e9ad64b6dc2)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:25.358938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:25.360919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 114/228] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:25 2022] GET /identity/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2 => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:25.367075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:25.424236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f60f4a-4bbc-4b73-9787-39a51009c439 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:25.515243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.515779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.516057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.516327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.517041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:25.517341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:25.518748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:25.522736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc1d1c-9b9d-4b1a-b8cc-daf5412589cb None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:25.539467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 115/229] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:25.545684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:25.599774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:25.603460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.604489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.604988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.605448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.613465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:25.615438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:25.617729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 115/230] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:25 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:25.625339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:25.648658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:25.671201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cdd2b-2c76-4d44-a0d0-ffc5b0d0e785 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:25.707682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.709099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.709503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.712777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] PATH_INFO: `/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.713857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=5cec43c4980d4c05815e0e9ad64b6dc2)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:25.716320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:25.720148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 116/231] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:25 2022] GET /identity/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:25.729815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:25.817844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024ddac-7b82-44bc-8501-20e5c4643124 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:25.897000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cdfc5-f434-499d-bb9d-d0873eb6f2c5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:25.913838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:25.917820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.918756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.919262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.919769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/5cec43c4980d4c05815e0e9ad64b6dc2/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.925055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a0f353314fa4f1f8740e6d18f9374c6, user_id=5cec43c4980d4c05815e0e9ad64b6dc2, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:25.929966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:25.960440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 116/232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:25 2022] PUT /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/5cec43c4980d4c05815e0e9ad64b6dc2/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:45:25.963945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba706e-a919-4edf-8cff-57f81bad0ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:25.964801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba706e-a919-4edf-8cff-57f81bad0ccd None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:25.965227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba706e-a919-4edf-8cff-57f81bad0ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:25.965645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba706e-a919-4edf-8cff-57f81bad0ccd None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:25.967083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba706e-a919-4edf-8cff-57f81bad0ccd None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:25.968900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba706e-a919-4edf-8cff-57f81bad0ccd None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:25.970869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 117/233] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:25 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:25.978319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:26.083822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:26.113621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032f541-3b41-45e7-b360-191d51308298 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:26.151981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c187e4b-ba80-4a9e-9079-d9ae7f6e2f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:26.153165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c187e4b-ba80-4a9e-9079-d9ae7f6e2f2c None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:26.153598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c187e4b-ba80-4a9e-9079-d9ae7f6e2f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:26.154017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c187e4b-ba80-4a9e-9079-d9ae7f6e2f2c None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/5cec43c4980d4c05815e0e9ad64b6dc2/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:26.156858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c187e4b-ba80-4a9e-9079-d9ae7f6e2f2c None admin] RBAC: Authorizing `identity:create_grant(project_id=8a0f353314fa4f1f8740e6d18f9374c6, user_id=5cec43c4980d4c05815e0e9ad64b6dc2, role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:26.158591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c187e4b-ba80-4a9e-9079-d9ae7f6e2f2c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:26.185152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 117/234] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:25 2022] PUT /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/5cec43c4980d4c05815e0e9ad64b6dc2/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:26.889651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d57b7-0773-4a90-8233-4c6dacd5d314 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:26.890539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d57b7-0773-4a90-8233-4c6dacd5d314 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:26.891321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d57b7-0773-4a90-8233-4c6dacd5d314 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:26.893107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 118/235] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:26.898854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4096fb7-13de-4dd0-9779-83e78ee7e82d None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:26.899607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4096fb7-13de-4dd0-9779-83e78ee7e82d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:26.900047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4096fb7-13de-4dd0-9779-83e78ee7e82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:26.928139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e4096fb7-13de-4dd0-9779-83e78ee7e82d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:26.929600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e4096fb7-13de-4dd0-9779-83e78ee7e82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:26.978960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f029b-c992-4abc-99fe-77c5616d0c16 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:26.979379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f029b-c992-4abc-99fe-77c5616d0c16 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:26.979745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f029b-c992-4abc-99fe-77c5616d0c16 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:26.981516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 119/236] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:26.986465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970cb8f-609e-4b95-9971-d1375ae9de0a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:26.986872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970cb8f-609e-4b95-9971-d1375ae9de0a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:26.987228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970cb8f-609e-4b95-9971-d1375ae9de0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:27.007082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-9970cb8f-609e-4b95-9971-d1375ae9de0a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:27.008644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9970cb8f-609e-4b95-9971-d1375ae9de0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:27.010951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 118/237] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:27.091326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 120/238] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:27.323898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4096fb7-13de-4dd0-9779-83e78ee7e82d None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:27.423668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:27.463812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4096fb7-13de-4dd0-9779-83e78ee7e82d None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:27.512981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:27.513957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:27.514433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:27.514925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:27.533304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:27.539309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:27.540190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:27.540997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:27.543508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 119/239] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:27.549468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970cb8f-609e-4b95-9971-d1375ae9de0a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:27.557645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:27.742974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970cb8f-609e-4b95-9971-d1375ae9de0a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:27.751582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:27.793466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcaa63-abf5-436e-b6a1-1c32d7c4bf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:27.794087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcaa63-abf5-436e-b6a1-1c32d7c4bf9b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:27.794365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcaa63-abf5-436e-b6a1-1c32d7c4bf9b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:27.794635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcaa63-abf5-436e-b6a1-1c32d7c4bf9b None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:27.795308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcaa63-abf5-436e-b6a1-1c32d7c4bf9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:27.795604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcaa63-abf5-436e-b6a1-1c32d7c4bf9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:27.797027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcaa63-abf5-436e-b6a1-1c32d7c4bf9b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:27.804185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3752a-ad84-41e2-94bc-42d76dc42ab6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:27.813445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 121/240] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:27.814675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc36b-9cd7-4783-a2e8-f25d9b6f562d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:27.814955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc36b-9cd7-4783-a2e8-f25d9b6f562d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:27.815219 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc36b-9cd7-4783-a2e8-f25d9b6f562d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:27.816415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 122/241] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:27.821281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc36b-9cd7-4783-a2e8-f25d9b6f562d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:27.840081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:27.848226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8effde-8ee3-4dec-9250-81e4454eeb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:27.848984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8effde-8ee3-4dec-9250-81e4454eeb00 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:27.849954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8effde-8ee3-4dec-9250-81e4454eeb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:27.850237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8effde-8ee3-4dec-9250-81e4454eeb00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:27.851612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8effde-8ee3-4dec-9250-81e4454eeb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:27.853808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8effde-8ee3-4dec-9250-81e4454eeb00 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:27.855939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 120/242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:27.861029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86647-ce95-4997-aae0-7a95aaa3d555 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:27.861029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86647-ce95-4997-aae0-7a95aaa3d555 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:27.861029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c86647-ce95-4997-aae0-7a95aaa3d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:27.880091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-32c86647-ce95-4997-aae0-7a95aaa3d555 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:27.881595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-32c86647-ce95-4997-aae0-7a95aaa3d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:27.951670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc36b-9cd7-4783-a2e8-f25d9b6f562d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:27.973017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 121/243] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:27.980869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86647-ce95-4997-aae0-7a95aaa3d555 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:28.020111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:28.021669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:28.022213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:28.022676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] PATH_INFO: `/v3/users/a9bfca3d06b9429c8088942ebd71b29a` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:28.050713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] RBAC: Authorizing `identity:get_user(user_id=a9bfca3d06b9429c8088942ebd71b29a)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:28.053337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:28.055311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 123/244] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:27 2022] GET /identity/v3/users/a9bfca3d06b9429c8088942ebd71b29a => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:28.060926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:28.092676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c86647-ce95-4997-aae0-7a95aaa3d555 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:28.117508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:28.150872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:28.163917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:28.164823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:28.165470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:28.169694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:28.171943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:28.173606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:28.222827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 122/245] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:45:27 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:28.265248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d536a8-f786-4092-a94d-d918e78ab79e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:28.339134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa131f82-a194-475d-bfe3-a7584e1be67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:28.339784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131f82-a194-475d-bfe3-a7584e1be67b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:28.340059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131f82-a194-475d-bfe3-a7584e1be67b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:28.340334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131f82-a194-475d-bfe3-a7584e1be67b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:28.430582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa131f82-a194-475d-bfe3-a7584e1be67b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:28.432490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa131f82-a194-475d-bfe3-a7584e1be67b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:28.472115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:28.504638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:28.578553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 124/246] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 15:45:28.579772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743dac1-1b3f-428e-b74d-2c1cab2d57d4 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:28.580639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743dac1-1b3f-428e-b74d-2c1cab2d57d4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:28.581007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743dac1-1b3f-428e-b74d-2c1cab2d57d4 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:28.582754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 125/247] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:28.587127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743dac1-1b3f-428e-b74d-2c1cab2d57d4 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:28.657937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb6418-2e55-492b-ba17-2abbd908f4ec None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:28.703830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6398f-1c32-4a5d-a33a-723fc299a72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:28.704476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6398f-1c32-4a5d-a33a-723fc299a72b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:28.704784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6398f-1c32-4a5d-a33a-723fc299a72b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:28.705057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6398f-1c32-4a5d-a33a-723fc299a72b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:28.719494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6398f-1c32-4a5d-a33a-723fc299a72b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-cbf6398f-1c32-4a5d-a33a-723fc299a72b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:28.727660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:28.728587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:28.729716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:28.732541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 123/248] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:28.732541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb15c7-fae8-4f1e-8d1e-64a0fdd5d024 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:28.732541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb15c7-fae8-4f1e-8d1e-64a0fdd5d024 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:28.732541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb15c7-fae8-4f1e-8d1e-64a0fdd5d024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:28.737716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743dac1-1b3f-428e-b74d-2c1cab2d57d4 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:28.758696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d4cb15c7-fae8-4f1e-8d1e-64a0fdd5d024 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:28.760243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d4cb15c7-fae8-4f1e-8d1e-64a0fdd5d024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:28.780056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:28.790031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:28.790681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:28.791126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:28.791519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:28.792895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:28.795554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:28.797840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 126/249] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:28 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:28.798647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:28.828339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:28.873490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 124/250] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:28.874281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb15c7-fae8-4f1e-8d1e-64a0fdd5d024 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:28.997444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb15c7-fae8-4f1e-8d1e-64a0fdd5d024 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:29.043801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9925b-96b8-42fc-9540-06254d29d130 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:29.073477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ad7ee-42a6-46e4-9b6c-3741a651a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:29.074506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad7ee-42a6-46e4-9b6c-3741a651a3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:29.075024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad7ee-42a6-46e4-9b6c-3741a651a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:29.075497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad7ee-42a6-46e4-9b6c-3741a651a3f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:29.076436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ad7ee-42a6-46e4-9b6c-3741a651a3f9 None admin] RBAC: Adding query filter params (role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=a9bfca3d06b9429c8088942ebd71b29a) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:29.076969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ad7ee-42a6-46e4-9b6c-3741a651a3f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:29.078671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ad7ee-42a6-46e4-9b6c-3741a651a3f9 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:29.089515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 125/251] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:28 2022] GET /identity/v3/role_assignments?user.id=a9bfca3d06b9429c8088942ebd71b29a&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:29.147505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d9edb-8c6e-4e7c-8734-b36b8dd18f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:29.148057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9edb-8c6e-4e7c-8734-b36b8dd18f35 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:29.148336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9edb-8c6e-4e7c-8734-b36b8dd18f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:29.148606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d9edb-8c6e-4e7c-8734-b36b8dd18f35 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:29.149337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d9edb-8c6e-4e7c-8734-b36b8dd18f35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:29.149686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d9edb-8c6e-4e7c-8734-b36b8dd18f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:29.151921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d9edb-8c6e-4e7c-8734-b36b8dd18f35 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:29.158467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d5c9a-c1b4-43a9-aade-0be83f6d4996 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:29.159078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d5c9a-c1b4-43a9-aade-0be83f6d4996 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:29.159575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d5c9a-c1b4-43a9-aade-0be83f6d4996 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:29.164855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 126/252] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:29.165834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 127/253] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:29.172480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d5c9a-c1b4-43a9-aade-0be83f6d4996 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:29.176034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b475-272c-4091-bd60-18388dd33abd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:29.176336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b475-272c-4091-bd60-18388dd33abd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:29.177215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b475-272c-4091-bd60-18388dd33abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:29.215812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-dfd0b475-272c-4091-bd60-18388dd33abd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:29.218350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dfd0b475-272c-4091-bd60-18388dd33abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:29.252738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:29.352288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:29.386539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 128/254] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:29.408275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b475-272c-4091-bd60-18388dd33abd None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:29.424098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d5c9a-c1b4-43a9-aade-0be83f6d4996 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:29.533438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:29.534544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:29.535021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:29.535484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] PATH_INFO: `/v3/users/db69d9c655df416d82282b0e261e1049` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:29.537108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] RBAC: Authorizing `identity:get_user(user_id=db69d9c655df416d82282b0e261e1049)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:29.538776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:29.540604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 127/255] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:29 2022] GET /identity/v3/users/db69d9c655df416d82282b0e261e1049 => generated 297 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:29.546243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:29.654643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b475-272c-4091-bd60-18388dd33abd None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:29.752396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:29.753001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:29.753279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:29.753548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:29.767344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a6bd2-58aa-47dd-a309-d01e22d36b86 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:29.777349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:29.781977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:29.782593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:29.783441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:29.784227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:29.788275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 129/256] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:29.789924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:29.835209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:29.835852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:29.836132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:29.836405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:29.837700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:29.839158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:29.840978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 128/257] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:29 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:29.846732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:29.881120 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:29.994499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ef7fe-2c51-489c-a23e-da0aaa667d4b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:29.996683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b7b2a-2a97-40ea-a037-f8a54a66e54d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:30.033291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:30.033814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:30.034209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:30.034553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:30.035286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:30.035659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:30.037758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:30.047942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 129/258] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:30.049015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:30.083235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:30.085002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:30.085496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:30.085928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:30.097530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:30.119110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:30.125419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:30.126281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:30.127138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:30.130802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 130/259] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:30.131656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:30.220007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be67e8-5e25-4443-b6a0-019b90178926 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:30.254787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:30.255524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:30.255805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:30.256074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/db69d9c655df416d82282b0e261e1049/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:30.258039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a0f353314fa4f1f8740e6d18f9374c6, user_id=db69d9c655df416d82282b0e261e1049, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:30.259514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:30.287529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 130/260] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:30 2022] PUT /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/db69d9c655df416d82282b0e261e1049/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:30.288286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:30.362967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1d14b-0d7b-46ef-81df-a5bf27da9bc1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:30.376155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:30.482951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:30.483591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:30.483872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:30.484147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] PATH_INFO: `/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:30.493645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] RBAC: Authorizing `identity:get_user(user_id=5cec43c4980d4c05815e0e9ad64b6dc2)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:30.495187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:30.497300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 131/261] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:30 2022] GET /identity/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2 => generated 283 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:30.501863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:30.502906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:30.514933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9a80c-7f37-4a9d-b3dd-1a0f276aab30 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:30.585216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:30.585856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:30.586406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:30.586679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:30.587340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:30.587641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:30.589067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:30.597837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 131/262] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:30.606407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:30.713850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b43ee-2263-4245-b037-22d274fcfc52 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:30.764413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b12342-9373-4ede-a5b9-b0d69a616c00 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:30.779482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:30.780109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:30.780388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:30.780657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:30.831226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:30.831880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:30.832161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:30.832439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] PATH_INFO: `/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:30.834498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] RBAC: Authorizing `identity:get_user(user_id=5cec43c4980d4c05815e0e9ad64b6dc2)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:30.836011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:30.841120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 132/263] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:30 2022] GET /identity/v3/users/5cec43c4980d4c05815e0e9ad64b6dc2 => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:30.843846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:30.867663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:30.890257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:30.912859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:30.914357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:31.014871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af051-ba8e-4685-bfe0-8808cc1bea70 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:31.058214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc2cb3-a4d7-4592-ad57-c5ff077205bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:31.058841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc2cb3-a4d7-4592-ad57-c5ff077205bc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.059113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc2cb3-a4d7-4592-ad57-c5ff077205bc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.059384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc2cb3-a4d7-4592-ad57-c5ff077205bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.104761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 132/264] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:31.111230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:31.237812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc2cb3-a4d7-4592-ad57-c5ff077205bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:31.240315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc2cb3-a4d7-4592-ad57-c5ff077205bc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:31.256846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:31.294686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bae168-7bf1-49e1-baec-edee349c60a7 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:31.298624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:31.383272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf312a5-8a09-4a80-8d6d-a44b1871bd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:31.383836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf312a5-8a09-4a80-8d6d-a44b1871bd34 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.384111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf312a5-8a09-4a80-8d6d-a44b1871bd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.384378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf312a5-8a09-4a80-8d6d-a44b1871bd34 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.385546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf312a5-8a09-4a80-8d6d-a44b1871bd34 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:31.387012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf312a5-8a09-4a80-8d6d-a44b1871bd34 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:31.388654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 133/265] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:31 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:31.391762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e7b3d-f9d6-4c3d-ba09-d55f63ac151d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.392041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e7b3d-f9d6-4c3d-ba09-d55f63ac151d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.392308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e7b3d-f9d6-4c3d-ba09-d55f63ac151d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.395207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 134/266] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:31.397203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e7b3d-f9d6-4c3d-ba09-d55f63ac151d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:31.434728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 133/267] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 15:45:31.441624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db47b29-b119-482d-868f-fb7c17833224 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.441915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db47b29-b119-482d-868f-fb7c17833224 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.442721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db47b29-b119-482d-868f-fb7c17833224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.489705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-2db47b29-b119-482d-868f-fb7c17833224 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:31.492112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2db47b29-b119-482d-868f-fb7c17833224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:31.601109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 134/268] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:31.601972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db47b29-b119-482d-868f-fb7c17833224 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:31.609935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e7b3d-f9d6-4c3d-ba09-d55f63ac151d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:31.651200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:31.651732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.652006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.652274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.653035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=5cec43c4980d4c05815e0e9ad64b6dc2) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:31.653339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:31.654812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:31.664045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 135/269] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:31 2022] GET /identity/v3/role_assignments?user.id=5cec43c4980d4c05815e0e9ad64b6dc2&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:31.689170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:31.758709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db47b29-b119-482d-868f-fb7c17833224 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:31.815153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17275-ff8a-4080-b1cd-9e1fdf247235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:31.815829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17275-ff8a-4080-b1cd-9e1fdf247235 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.816105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17275-ff8a-4080-b1cd-9e1fdf247235 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.816372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17275-ff8a-4080-b1cd-9e1fdf247235 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.818013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17275-ff8a-4080-b1cd-9e1fdf247235 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:31.819535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17275-ff8a-4080-b1cd-9e1fdf247235 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:31.821644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 135/270] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:31 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:31.827354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:31.842028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:31.875866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8889f5-0083-49f9-b716-1531a410c6bd None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.876174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8889f5-0083-49f9-b716-1531a410c6bd None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.876408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8889f5-0083-49f9-b716-1531a410c6bd None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.877932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 136/271] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:31.884011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56d06-fc4c-443c-8359-d2114e8252d5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:31.884289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56d06-fc4c-443c-8359-d2114e8252d5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:31.884565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c56d06-fc4c-443c-8359-d2114e8252d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:31.905476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e7c56d06-fc4c-443c-8359-d2114e8252d5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:31.906606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e7c56d06-fc4c-443c-8359-d2114e8252d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:31.977914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff118613-646a-4924-9132-533e72dacf66 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.027991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:32.028790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.029356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.029630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.030369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] RBAC: Adding query filter params (role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=5cec43c4980d4c05815e0e9ad64b6dc2) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:32.030676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:32.032566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:32.042807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 136/272] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:31 2022] GET /identity/v3/role_assignments?user.id=5cec43c4980d4c05815e0e9ad64b6dc2&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:32.043933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.046778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 137/273] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:32.144768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:32.181527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1429eb6-09a6-468c-8cc1-4f2eb150c6c5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.228604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362203-82df-4cd8-99f6-660ac5464240 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.229679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362203-82df-4cd8-99f6-660ac5464240 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.230152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c362203-82df-4cd8-99f6-660ac5464240 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.231732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 138/274] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:32.238267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:32.238465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fba95-3baa-4f69-b48b-985fcf742895 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.238720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fba95-3baa-4f69-b48b-985fcf742895 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.239508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fba95-3baa-4f69-b48b-985fcf742895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.239959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.240294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.240569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.263378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e61fba95-3baa-4f69-b48b-985fcf742895 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:32.264193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:32.264631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e61fba95-3baa-4f69-b48b-985fcf742895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:32.269758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:32.270906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:32.271685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:32.275911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 137/275] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:32.280779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.377125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:32.398783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 139/276] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:32.419396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fba95-3baa-4f69-b48b-985fcf742895 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.452278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e8c8a-e658-4eb1-9968-148180245fe9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.490907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:32.499618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:32.500535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.501039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.501598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.502479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:32.502979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:32.504788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:32.515581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 138/277] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:32.523377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.549448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fba95-3baa-4f69-b48b-985fcf742895 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.609592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:32.610303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.610584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.610848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.611992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:32.613817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:32.615434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 140/278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:32.624395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.647733 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628c6ff-d8d7-4905-9e17-75c1982cfa23 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.690547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:32.698503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:32.699140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.699413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.699682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] PATH_INFO: `/v3/users/7eb6de16c2984fbd91f9cd997d356450` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.713491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=7eb6de16c2984fbd91f9cd997d356450)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:32.714905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:32.716688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 139/279] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:32 2022] GET /identity/v3/users/7eb6de16c2984fbd91f9cd997d356450 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:32.717321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.746971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b2cc1-426d-4dbd-8305-84abf01887f5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.769634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:32.784330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:32.784974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.785253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.785535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.786950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:32.788398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:32.816783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 141/280] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:45:32 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:32.816783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.837588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53183a-f3ad-4ba9-a52d-f05763a64cb2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.891228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:32.892187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:32.892797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:32.893263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:32.909138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:32.916267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:32.917717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:32.918712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:32.920859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 140/281] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:32.926944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:32.947637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25335a0-ef35-4760-8a3f-aa46fd13b2b2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:32.961640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:33.005868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:33.006425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:33.007149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:33.007436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:33.008508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, user.id=7eb6de16c2984fbd91f9cd997d356450, scope.system=all) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:33.008840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:33.010373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:33.019514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 142/282] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 15:45:32 2022] GET /identity/v3/role_assignments?user.id=7eb6de16c2984fbd91f9cd997d356450&scope.system=all&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:33.059304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8513714-f789-4d88-9c2d-45e11107fd91 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:33.072497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:33.109281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:33.109897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:33.110169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:33.110436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:33.111180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:33.111482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:33.113050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:33.123863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 141/283] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:33.130031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:33.248152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964fee-dbc9-45c0-a71e-f55b23fa0899 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:33.292364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0153-82ab-4072-83ae-90f4d454c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:33.293324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0153-82ab-4072-83ae-90f4d454c55d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:33.293741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0153-82ab-4072-83ae-90f4d454c55d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:33.294145 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0153-82ab-4072-83ae-90f4d454c55d None admin] PATH_INFO: `/v3/users/a9bfca3d06b9429c8088942ebd71b29a` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:33.296446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0153-82ab-4072-83ae-90f4d454c55d None admin] RBAC: Authorizing `identity:get_user(user_id=a9bfca3d06b9429c8088942ebd71b29a)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:33.298113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0153-82ab-4072-83ae-90f4d454c55d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:33.299873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 143/284] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:33 2022] GET /identity/v3/users/a9bfca3d06b9429c8088942ebd71b29a => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:33.306126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:33.402955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79a121-1fcc-4c81-97cc-29b55030b5c4 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:33.403510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79a121-1fcc-4c81-97cc-29b55030b5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:33.403931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79a121-1fcc-4c81-97cc-29b55030b5c4 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:33.405741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 144/285] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:33.410798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0844bb9-c2ac-4dee-b848-0a3fcc62d31c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:33.411315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0844bb9-c2ac-4dee-b848-0a3fcc62d31c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:33.411917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0844bb9-c2ac-4dee-b848-0a3fcc62d31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:33.439277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d0844bb9-c2ac-4dee-b848-0a3fcc62d31c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:33.440478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d0844bb9-c2ac-4dee-b848-0a3fcc62d31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:33.481159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:33.482638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea6479-add0-4195-89a1-cfdf7d1d59b8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:33.532206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 145/286] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:33.546790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:33.547359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:33.547635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:33.547907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:33.553429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:33.554915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:33.556997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 142/287] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:33 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:33.562634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0844bb9-c2ac-4dee-b848-0a3fcc62d31c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:33.676265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0844bb9-c2ac-4dee-b848-0a3fcc62d31c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:33.726367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:33.727414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:33.727890 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:33.728370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/a9bfca3d06b9429c8088942ebd71b29a/roles/5182eb7e9f3a4b9b8d563e8bf9888d0f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:33.737465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a0f353314fa4f1f8740e6d18f9374c6, user_id=a9bfca3d06b9429c8088942ebd71b29a, role_id=5182eb7e9f3a4b9b8d563e8bf9888d0f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:33.739281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:33.763647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 146/288] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:45:33 2022] PUT /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/users/a9bfca3d06b9429c8088942ebd71b29a/roles/5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:33.810018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:33.874860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:33.959453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50b0e0-655f-4ffe-b4b7-f54b1b78b32d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:34.007940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:34.008586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:34.008908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:34.009184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:34.022649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:34.027970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:34.028879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:34.029803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 143/289] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:34.035482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:34.077529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:34.158225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5aa8-cb40-4824-b226-f228c9a633f0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:34.200143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:34.201188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:34.201664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:34.202127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:34.203014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:34.203546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:34.205206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:34.220748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 147/290] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:34.221562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:34.385108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:34.387973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8ce6f-9211-484d-9aeb-7f00cfa861a4 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:34.435080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:34.435713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:34.435992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:34.436264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] PATH_INFO: `/v3/users/db69d9c655df416d82282b0e261e1049` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:34.437444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] RBAC: Authorizing `identity:get_user(user_id=db69d9c655df416d82282b0e261e1049)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:34.438893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:34.440786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 144/291] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:34 2022] GET /identity/v3/users/db69d9c655df416d82282b0e261e1049 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:34.446809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:34.562309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:34.573495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d69b6-665f-44e2-9ff5-bd91a7043ecf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:34.616851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:34.617817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:34.618287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:34.618745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:34.698643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:34.700414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:34.810564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:34.822120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 148/292] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:34.827547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:34.983136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340bfdf-561e-417f-95b3-fa0ddafdf46f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:35.027358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7a934-dcba-4214-9627-0c10afc9fe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:35.027989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7a934-dcba-4214-9627-0c10afc9fe5b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.028261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7a934-dcba-4214-9627-0c10afc9fe5b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.028532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7a934-dcba-4214-9627-0c10afc9fe5b None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:35.030645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7a934-dcba-4214-9627-0c10afc9fe5b None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:35.032173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7a934-dcba-4214-9627-0c10afc9fe5b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:35.034005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 145/293] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:34 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:35.040152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:35.187898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca60ce-c1f0-4cd6-9016-cb74a1dd8c14 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:35.236358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d405616-84c6-4d85-b024-cf9c9feaf134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:35.237327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d405616-84c6-4d85-b024-cf9c9feaf134 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.237797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d405616-84c6-4d85-b024-cf9c9feaf134 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.238256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d405616-84c6-4d85-b024-cf9c9feaf134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:35.239292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d405616-84c6-4d85-b024-cf9c9feaf134 None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=db69d9c655df416d82282b0e261e1049) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:35.239825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d405616-84c6-4d85-b024-cf9c9feaf134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:35.241469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d405616-84c6-4d85-b024-cf9c9feaf134 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:35.253555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 149/294] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:35 2022] GET /identity/v3/role_assignments?user.id=db69d9c655df416d82282b0e261e1049&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:35.749491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7648b29-c8fc-4a2d-ba5e-ce67280ca323 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.750530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7648b29-c8fc-4a2d-ba5e-ce67280ca323 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.751021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7648b29-c8fc-4a2d-ba5e-ce67280ca323 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:35.752886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 146/295] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:35.758737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c2190-a6b7-44e8-afe2-c1ffb42a1018 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.759564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c2190-a6b7-44e8-afe2-c1ffb42a1018 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.760062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c2190-a6b7-44e8-afe2-c1ffb42a1018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:35.778795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-3f6c2190-a6b7-44e8-afe2-c1ffb42a1018 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:35.780299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3f6c2190-a6b7-44e8-afe2-c1ffb42a1018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:35.832302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fe94b-4843-4cd0-9832-f6f3838713ef None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.832855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fe94b-4843-4cd0-9832-f6f3838713ef None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.833258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fe94b-4843-4cd0-9832-f6f3838713ef None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:35.834946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 147/296] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:35.840752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:35.840938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899086a0-702c-4a6b-b9e5-997c6b5b6039 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.841146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899086a0-702c-4a6b-b9e5-997c6b5b6039 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.841343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899086a0-702c-4a6b-b9e5-997c6b5b6039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:35.862895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 150/297] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:35.871725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-899086a0-702c-4a6b-b9e5-997c6b5b6039 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:35.872953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-899086a0-702c-4a6b-b9e5-997c6b5b6039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:35.904552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:35.947166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 148/298] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:35.968480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc176369-7c7c-4030-a7f4-6d6b0c196279 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.968836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc176369-7c7c-4030-a7f4-6d6b0c196279 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.969063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc176369-7c7c-4030-a7f4-6d6b0c196279 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:35.970661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 151/299] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:35.977527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016157d0-48bf-4ba7-915f-e2d29dc8439f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:35.977778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016157d0-48bf-4ba7-915f-e2d29dc8439f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:35.977990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016157d0-48bf-4ba7-915f-e2d29dc8439f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.001907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-016157d0-48bf-4ba7-915f-e2d29dc8439f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:36.003012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-016157d0-48bf-4ba7-915f-e2d29dc8439f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:36.076811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 149/300] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:36.156154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc176369-7c7c-4030-a7f4-6d6b0c196279 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.238105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016157d0-48bf-4ba7-915f-e2d29dc8439f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.265290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc176369-7c7c-4030-a7f4-6d6b0c196279 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.308530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46704e2-b661-406b-85df-5c3242031c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:36.309203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46704e2-b661-406b-85df-5c3242031c64 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.309478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46704e2-b661-406b-85df-5c3242031c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.309748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46704e2-b661-406b-85df-5c3242031c64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.310860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46704e2-b661-406b-85df-5c3242031c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:36.312305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46704e2-b661-406b-85df-5c3242031c64 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:36.324310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 152/301] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:36.326342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6af4e0-6ce3-4813-99dc-6f4630abd5b4 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.326630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6af4e0-6ce3-4813-99dc-6f4630abd5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.326894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6af4e0-6ce3-4813-99dc-6f4630abd5b4 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.328312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 153/302] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:36.328955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6af4e0-6ce3-4813-99dc-6f4630abd5b4 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.341206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:36.347792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016157d0-48bf-4ba7-915f-e2d29dc8439f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.384678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93ee57-c944-4cdf-882a-a9d64f9cb32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:36.385368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93ee57-c944-4cdf-882a-a9d64f9cb32e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.385642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93ee57-c944-4cdf-882a-a9d64f9cb32e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.385910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93ee57-c944-4cdf-882a-a9d64f9cb32e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.387865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:36.396930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93ee57-c944-4cdf-882a-a9d64f9cb32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-6c93ee57-c944-4cdf-882a-a9d64f9cb32e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:36.401738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:36.402599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:36.403582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 150/303] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:36.405229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a80d2c-4826-4532-9ee7-f580bd33b3e1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.405518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a80d2c-4826-4532-9ee7-f580bd33b3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.405789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a80d2c-4826-4532-9ee7-f580bd33b3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.423681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-96a80d2c-4826-4532-9ee7-f580bd33b3e1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:36.424914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-96a80d2c-4826-4532-9ee7-f580bd33b3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:36.435200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6af4e0-6ce3-4813-99dc-6f4630abd5b4 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.468339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:36.468960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.469230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.469494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] PATH_INFO: `/v3/groups` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.470115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:36.471531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:36.486532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 154/304] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Oct 3 15:45:36 2022] POST /identity/v3/groups => generated 229 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:45:36.487340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.501643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 151/305] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:36.502997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a80d2c-4826-4532-9ee7-f580bd33b3e1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.612070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c200b4-2b09-4068-87c0-7e656ed56076 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.613318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:36.630909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:36.633226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a80d2c-4826-4532-9ee7-f580bd33b3e1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.655278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:36.656349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.656790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.657303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.670419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:36.676163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:36.677212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:36.677997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:36.678610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:36.678610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.678748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 155/306] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:36.678811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.678811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.678811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:36.678811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:36.680153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:36.687429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.689739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 152/307] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:36.695578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.808112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbfe28-b2c6-4643-8ae9-8cfa4039d8f8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.813407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7856e-a4ee-449a-a21b-22ab66943963 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.846085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:36.846958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.847236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.847899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] PATH_INFO: `/v3/users/7eb6de16c2984fbd91f9cd997d356450` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.849543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] RBAC: Authorizing `identity:get_user(user_id=7eb6de16c2984fbd91f9cd997d356450)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:36.850989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:36.857180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 153/308] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:36 2022] GET /identity/v3/users/7eb6de16c2984fbd91f9cd997d356450 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:36.857330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:36.857330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:36.857517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.857586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:36.857586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:36.857586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:36.857586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:36.857586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:36.865724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 156/309] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:36.866308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:36.879791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:36.906907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:36.965762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e393c-3a87-4fc3-8dca-f5c16153fc88 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:36.975530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75ba2-7441-4ec4-a221-ee694c207408 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:37.000348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.001037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.001309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.001577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.011164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.011806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.012080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.012235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:37.012384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] PATH_INFO: `/v3/system/users/7eb6de16c2984fbd91f9cd997d356450/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.013772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7eb6de16c2984fbd91f9cd997d356450, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:37.015135 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:37.017448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:37.018284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:37.019090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 154/310] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:37.019719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:37.029209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 157/311] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Oct 3 15:45:36 2022] PUT /identity/v3/system/users/7eb6de16c2984fbd91f9cd997d356450/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:37.029935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:37.120220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:37.152331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d82a10-4301-4a67-b56a-4f41002dc075 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:37.172488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:37.176858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aec9ac-a011-46d9-a7e8-700067da9716 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:37.196381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.197197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.197572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.198331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] PATH_INFO: `/v3/users/dff9d86ed8b047ee85ff610e17d01a7f` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.218316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] RBAC: Authorizing `identity:get_user(user_id=dff9d86ed8b047ee85ff610e17d01a7f)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:37.219818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:37.225963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 155/312] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:37 2022] GET /identity/v3/users/dff9d86ed8b047ee85ff610e17d01a7f => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:37.228532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:37.228948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.229524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.229803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.230076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.230746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:37.231049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:37.232417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:37.244639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 158/313] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:37.257779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:37.342862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55378f63-d0ee-4afc-9f0e-7979314dcbfd None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:37.380539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.381536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.381536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.381916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.382710 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] RBAC: Adding query filter params (role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f, user.id=dff9d86ed8b047ee85ff610e17d01a7f, scope.system=all) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:37.383088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:37.384692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:37.386778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:37.391979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 156/314] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 15:45:37 2022] GET /identity/v3/role_assignments?user.id=dff9d86ed8b047ee85ff610e17d01a7f&scope.system=all&role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:37.404227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dbd00-e0e8-4423-9360-36cc950b3299 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:37.444571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.445300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.445640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.447090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] PATH_INFO: `/v3/users/a9bfca3d06b9429c8088942ebd71b29a` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.450080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] RBAC: Authorizing `identity:get_user(user_id=a9bfca3d06b9429c8088942ebd71b29a)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:37.452697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:37.454850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 159/315] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:37 2022] GET /identity/v3/users/a9bfca3d06b9429c8088942ebd71b29a => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:37.461616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:37.579487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e35a2-aec7-44f8-bc34-f0463173f657 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:37.619109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.620063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.620563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.621054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.688920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:37.693575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:37.695592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:37.803819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 157/316] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:37.809125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:37.844843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:37.924040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38811d6f-7669-41b5-b8b0-987adc9038a8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:37.962124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adcb32-781e-4b01-93ab-ef9a2ad2571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:37.963079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adcb32-781e-4b01-93ab-ef9a2ad2571e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:37.963622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adcb32-781e-4b01-93ab-ef9a2ad2571e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:37.964080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adcb32-781e-4b01-93ab-ef9a2ad2571e None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:37.965419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adcb32-781e-4b01-93ab-ef9a2ad2571e None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:37.967078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77adcb32-781e-4b01-93ab-ef9a2ad2571e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:37.969312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 160/317] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:37 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:37.976806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:38.076943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:38.081717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5466b2-c795-4757-80af-8fbc6176366a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:38.123029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b8f81-0581-4a4e-bd62-f04cea8bceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:38.123637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b8f81-0581-4a4e-bd62-f04cea8bceeb None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:38.123913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b8f81-0581-4a4e-bd62-f04cea8bceeb None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:38.124181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b8f81-0581-4a4e-bd62-f04cea8bceeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:38.124974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b8f81-0581-4a4e-bd62-f04cea8bceeb None admin] RBAC: Adding query filter params (role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6, user.id=a9bfca3d06b9429c8088942ebd71b29a) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:38.125286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b8f81-0581-4a4e-bd62-f04cea8bceeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:38.126742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b8f81-0581-4a4e-bd62-f04cea8bceeb None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:38.133830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 158/318] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:45:37 2022] GET /identity/v3/role_assignments?user.id=a9bfca3d06b9429c8088942ebd71b29a&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:38.843152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71575a01-513a-4332-b5fc-37e34598a790 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:38.843518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71575a01-513a-4332-b5fc-37e34598a790 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:38.843725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71575a01-513a-4332-b5fc-37e34598a790 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:38.845109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 161/319] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:38.851164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e40e23-2e48-4c8d-90f6-7062a85de1e0 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:38.851531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e40e23-2e48-4c8d-90f6-7062a85de1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:38.851868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e40e23-2e48-4c8d-90f6-7062a85de1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:38.870370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c5e40e23-2e48-4c8d-90f6-7062a85de1e0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:38.871535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c5e40e23-2e48-4c8d-90f6-7062a85de1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:38.941267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 159/320] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:39.197190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4dd8a-e4ba-41e6-8af8-6a58b42bf00a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.197485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4dd8a-e4ba-41e6-8af8-6a58b42bf00a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.197665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4dd8a-e4ba-41e6-8af8-6a58b42bf00a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.199161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 162/321] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:39.204240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77195a-630f-451a-a89a-b1e2215b91eb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.204685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77195a-630f-451a-a89a-b1e2215b91eb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.204685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77195a-630f-451a-a89a-b1e2215b91eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.205112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4dd8a-e4ba-41e6-8af8-6a58b42bf00a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:39.225739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-db77195a-630f-451a-a89a-b1e2215b91eb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:39.226911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-db77195a-630f-451a-a89a-b1e2215b91eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:39.294624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 160/322] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:39.313333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:39.320913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4dd8a-e4ba-41e6-8af8-6a58b42bf00a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:39.356182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:39.356843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.357118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.357382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.358460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:39.359898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:39.361687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 163/323] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:39.367507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77195a-630f-451a-a89a-b1e2215b91eb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:39.392390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:39.480208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77195a-630f-451a-a89a-b1e2215b91eb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:39.516350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24830c95-ea14-49e6-bd39-0de06efb0387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:39.517024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830c95-ea14-49e6-bd39-0de06efb0387 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.517295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830c95-ea14-49e6-bd39-0de06efb0387 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.517557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24830c95-ea14-49e6-bd39-0de06efb0387 None admin] PATH_INFO: `/v3/groups` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.518200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24830c95-ea14-49e6-bd39-0de06efb0387 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:39.520555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24830c95-ea14-49e6-bd39-0de06efb0387 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:39.540116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 161/324] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Oct 3 15:45:39 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:45:39.565168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:39.648509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a31d9-8b53-4fd5-8151-63f47009e83b None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.648955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a31d9-8b53-4fd5-8151-63f47009e83b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.649303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a31d9-8b53-4fd5-8151-63f47009e83b None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.653321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 162/325] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:39.656969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806844-9dcb-4625-9a2c-196a18e1ae2b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.657514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806844-9dcb-4625-9a2c-196a18e1ae2b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.657889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af806844-9dcb-4625-9a2c-196a18e1ae2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.677984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-af806844-9dcb-4625-9a2c-196a18e1ae2b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:39.678970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bb654-ca1b-469e-88aa-9304054a16ce None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:39.679311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-af806844-9dcb-4625-9a2c-196a18e1ae2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:39.717396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:39.718041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.718318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.718855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.732825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:39.738064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:39.739531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:39.740819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 164/326] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:39.745923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:39.760265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 163/327] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:39.777437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:39.856427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eefa3-134e-4281-8d1c-0c3482ecf4b4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:39.892284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:39.892930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:39.893211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:39.893480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:39.894142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:39.894444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:39.896627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:39.905565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 165/328] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:39.911445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806844-9dcb-4625-9a2c-196a18e1ae2b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:39.931126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:40.032418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af806844-9dcb-4625-9a2c-196a18e1ae2b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:40.055616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:40.080412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:40.081088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:40.081361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:40.081626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] PATH_INFO: `/v3/users/7eb6de16c2984fbd91f9cd997d356450` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:40.083644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=7eb6de16c2984fbd91f9cd997d356450)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:40.085171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:40.087549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 164/329] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:39 2022] GET /identity/v3/users/7eb6de16c2984fbd91f9cd997d356450 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:40.093396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:40.163138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:40.176378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17afb2-e5c0-469e-8d93-534b115cc1d0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:40.195919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:40.217350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:40.217988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:40.218263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:40.218530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:40.219821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a840b89-2252-4310-9c6e-442f3f005ea9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:40.230523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:40.235782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:40.236652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:40.237486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 166/330] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:40.243071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:40.265722 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:40.266427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:40.266694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:40.267083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:40.267924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] RBAC: Adding query filter params (role.id=5a57b9470d8a4718a12b9e65777a95df, user.id=7eb6de16c2984fbd91f9cd997d356450, scope.system=all) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:40.268356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:40.269877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:40.283911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 165/331] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 15:45:40 2022] GET /identity/v3/role_assignments?user.id=7eb6de16c2984fbd91f9cd997d356450&scope.system=all&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:40.366764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a492f-6223-45f8-8034-305cfd800392 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:40.407026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:40.407978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:40.408431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:40.408905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:40.409806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:40.410304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:40.412499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:40.422670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 167/332] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:40.428833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:40.559038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14425046-279c-493e-ae9d-8151ea36eb8f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:40.599248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13412c9-4e08-4017-b11c-4aa22f957cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:40.599884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13412c9-4e08-4017-b11c-4aa22f957cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:40.600153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13412c9-4e08-4017-b11c-4aa22f957cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:40.600414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13412c9-4e08-4017-b11c-4aa22f957cc2 None admin] PATH_INFO: `/v3/users/dff9d86ed8b047ee85ff610e17d01a7f` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:40.601615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13412c9-4e08-4017-b11c-4aa22f957cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=dff9d86ed8b047ee85ff610e17d01a7f)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:40.603077 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13412c9-4e08-4017-b11c-4aa22f957cc2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:40.604658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 166/333] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:40 2022] GET /identity/v3/users/dff9d86ed8b047ee85ff610e17d01a7f => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:40.610260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:40.624125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:40.729188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bcc67-f815-4d0b-98e8-3b1381a17f80 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:40.764253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4fa34-de4a-4826-ad73-ab490b7a159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:40.764945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4fa34-de4a-4826-ad73-ab490b7a159a None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:40.765218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4fa34-de4a-4826-ad73-ab490b7a159a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:40.765492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4fa34-de4a-4826-ad73-ab490b7a159a None admin] PATH_INFO: `/v3/system/users/dff9d86ed8b047ee85ff610e17d01a7f/roles/5182eb7e9f3a4b9b8d563e8bf9888d0f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:40.766899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4fa34-de4a-4826-ad73-ab490b7a159a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dff9d86ed8b047ee85ff610e17d01a7f, role_id=5182eb7e9f3a4b9b8d563e8bf9888d0f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:40.768265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4fa34-de4a-4826-ad73-ab490b7a159a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:40.776780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 168/334] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Oct 3 15:45:40 2022] PUT /identity/v3/system/users/dff9d86ed8b047ee85ff610e17d01a7f/roles/5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:42.177742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b0628-0d0d-45fd-9e56-13b6f47f4671 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:42.178257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b0628-0d0d-45fd-9e56-13b6f47f4671 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:42.178257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b0628-0d0d-45fd-9e56-13b6f47f4671 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:42.179948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 167/335] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:42.185213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6c714-5a33-4ca0-9625-9f10bac65b95 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:42.185673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6c714-5a33-4ca0-9625-9f10bac65b95 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:42.185745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6c714-5a33-4ca0-9625-9f10bac65b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:42.209951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-14c6c714-5a33-4ca0-9625-9f10bac65b95 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:42.211200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-14c6c714-5a33-4ca0-9625-9f10bac65b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:42.300171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 169/336] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:42.340658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2b786-f2a3-4400-9505-b609ceae1a87 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:42.341019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2b786-f2a3-4400-9505-b609ceae1a87 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:42.341276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2b786-f2a3-4400-9505-b609ceae1a87 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:42.342614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 168/337] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:42.348944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b86ed-58ab-4684-9cb7-b2ed1bd1799d None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:42.349581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b86ed-58ab-4684-9cb7-b2ed1bd1799d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:42.349854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b86ed-58ab-4684-9cb7-b2ed1bd1799d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:42.361092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:42.380748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e56b86ed-58ab-4684-9cb7-b2ed1bd1799d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:42.381934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e56b86ed-58ab-4684-9cb7-b2ed1bd1799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:42.470423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 170/338] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:42.636312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2b786-f2a3-4400-9505-b609ceae1a87 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:42.647765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:42.755688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2b786-f2a3-4400-9505-b609ceae1a87 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:42.790814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:42.791450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:42.791719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:42.791980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:42.806587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:42.811419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:42.812273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:42.813097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 169/339] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:42.820400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b86ed-58ab-4684-9cb7-b2ed1bd1799d None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:42.930163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b86ed-58ab-4684-9cb7-b2ed1bd1799d None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:42.972084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:42.972784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:42.973160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:42.973427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:42.974557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:42.974855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:42.976305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:42.977958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:42.993781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 171/340] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:42.999660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:43.009346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:43.123612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:43.184191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d151f9-8a99-4a6a-b000-82b9e6799550 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:43.185981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031afcf-d2d5-4362-b4e5-39841ea15ce0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:43.241630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:43.242370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:43.242715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:43.242894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:43.243966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:43.244638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:43.245061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:43.245335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:43.258623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:43.264051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:43.264991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:43.265816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:43.267012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 172/341] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:43.274072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:43.351688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:43.353248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:43.386101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:43.441097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:43.491027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 170/342] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:43.497210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:43.510803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea8cc2-db50-45dc-887a-b80b073ddb21 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:43.595590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:43.596209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:43.596482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:43.599529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:43.600139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:43.600440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:43.602447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:43.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 173/343] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:43.618424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:43.662169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be5560-9845-4d32-9415-05c0ffbcf200 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:43.741598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8451126-a603-4a24-ae93-c5d81e7ef60f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:43.749226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad935f5a-32d4-4459-b4a5-8a521ba1cb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:43.749886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad935f5a-32d4-4459-b4a5-8a521ba1cb39 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:43.750204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad935f5a-32d4-4459-b4a5-8a521ba1cb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:43.750472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad935f5a-32d4-4459-b4a5-8a521ba1cb39 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:43.751705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad935f5a-32d4-4459-b4a5-8a521ba1cb39 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:43.753304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad935f5a-32d4-4459-b4a5-8a521ba1cb39 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:43.755073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 171/344] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:43 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:43.759836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342be17-fd97-4d9d-84cc-a348f871e808 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:43.760261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:43.761005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342be17-fd97-4d9d-84cc-a348f871e808 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:43.761353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342be17-fd97-4d9d-84cc-a348f871e808 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:43.763033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 172/345] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:43.763765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342be17-fd97-4d9d-84cc-a348f871e808 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:43.787240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa85ba3-bb1e-4a58-a8e8-7784e2c76c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:43.787854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa85ba3-bb1e-4a58-a8e8-7784e2c76c33 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:43.788126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa85ba3-bb1e-4a58-a8e8-7784e2c76c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:43.788392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa85ba3-bb1e-4a58-a8e8-7784e2c76c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:43.840625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:43.920927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa85ba3-bb1e-4a58-a8e8-7784e2c76c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:43.922699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa85ba3-bb1e-4a58-a8e8-7784e2c76c33 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:43.992079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342be17-fd97-4d9d-84cc-a348f871e808 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:44.082915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5300f8-8119-4e60-9182-e5406017e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:44.083937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5300f8-8119-4e60-9182-e5406017e715 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.084476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5300f8-8119-4e60-9182-e5406017e715 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.084990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5300f8-8119-4e60-9182-e5406017e715 None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.098602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5300f8-8119-4e60-9182-e5406017e715 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:44.103068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5300f8-8119-4e60-9182-e5406017e715 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:44.108109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 173/346] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:43 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:44.114038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b93e7-4783-4ff9-8c07-c1a7ee6fec42 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.114558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b93e7-4783-4ff9-8c07-c1a7ee6fec42 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.115030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b93e7-4783-4ff9-8c07-c1a7ee6fec42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.127054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 174/347] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:44.128345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de8ddc-49b9-4623-844e-28431756d369 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.128625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de8ddc-49b9-4623-844e-28431756d369 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.132996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de8ddc-49b9-4623-844e-28431756d369 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.137657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 175/348] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:44.138488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de8ddc-49b9-4623-844e-28431756d369 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:44.140630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ed3b93e7-4783-4ff9-8c07-c1a7ee6fec42 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:44.142113 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ed3b93e7-4783-4ff9-8c07-c1a7ee6fec42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:44.162181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:44.240840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 174/349] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:44.240840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b93e7-4783-4ff9-8c07-c1a7ee6fec42 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:44.293686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:44.300472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de8ddc-49b9-4623-844e-28431756d369 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:44.367951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f68877-b927-4248-8342-7622c3a28963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:44.368563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f68877-b927-4248-8342-7622c3a28963 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.368907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f68877-b927-4248-8342-7622c3a28963 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.369148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f68877-b927-4248-8342-7622c3a28963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.369881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f68877-b927-4248-8342-7622c3a28963 None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=d7ac2137e95a4380a28ffb6603712f74) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:44.370181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f68877-b927-4248-8342-7622c3a28963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:44.371800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f68877-b927-4248-8342-7622c3a28963 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:44.378578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 176/350] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:44 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:44.380571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d79af-88b4-47df-baac-e83dcf33bf45 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.380880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d79af-88b4-47df-baac-e83dcf33bf45 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.381151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d79af-88b4-47df-baac-e83dcf33bf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.402291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-939d79af-88b4-47df-baac-e83dcf33bf45 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:44.403502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-939d79af-88b4-47df-baac-e83dcf33bf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:44.411257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b93e7-4783-4ff9-8c07-c1a7ee6fec42 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:44.454249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383335a-3871-4e25-a932-7dc6ac04a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:44.454985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383335a-3871-4e25-a932-7dc6ac04a084 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.455319 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383335a-3871-4e25-a932-7dc6ac04a084 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.455595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383335a-3871-4e25-a932-7dc6ac04a084 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.456916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383335a-3871-4e25-a932-7dc6ac04a084 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:44.459278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383335a-3871-4e25-a932-7dc6ac04a084 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:44.463802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 175/351] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:44 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:44.465183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a40b73-5dad-42db-a483-0ba4e48ae5c6 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.465466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a40b73-5dad-42db-a483-0ba4e48ae5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.465737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a40b73-5dad-42db-a483-0ba4e48ae5c6 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.467323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 176/352] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:44.474302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a40b73-5dad-42db-a483-0ba4e48ae5c6 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:44.489640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 177/353] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:44.491151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2865f7-df5c-479d-94d0-66396a96d478 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.491376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2865f7-df5c-479d-94d0-66396a96d478 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.491644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2865f7-df5c-479d-94d0-66396a96d478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.511204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-db2865f7-df5c-479d-94d0-66396a96d478 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:44.512389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-db2865f7-df5c-479d-94d0-66396a96d478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:44.530544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:44.599504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 178/354] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:44.662866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2865f7-df5c-479d-94d0-66396a96d478 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:44.703075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:44.727115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a40b73-5dad-42db-a483-0ba4e48ae5c6 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:44.798369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2865f7-df5c-479d-94d0-66396a96d478 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:44.805440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:44.806094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.806369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.806637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.807814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:44.809320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:44.811209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 177/355] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:44 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:44.818158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:44.837010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:44.837703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:44.837984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:44.838301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:44.851188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:44.857008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:44.858107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:44.859103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 179/356] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:44.865540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:44.872149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:44.993864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439aa43-ecd1-4aae-9082-6d49069df1e9 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:45.011928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99762c5-0ea1-404e-8f91-b1ad9fdef961 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:45.034345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:45.035023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.035383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.035649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.036320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:45.036617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:45.038230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:45.053670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 180/357] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:45.054468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:45.073479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:45.074096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.074373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.074640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.075381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=d7ac2137e95a4380a28ffb6603712f74) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:45.075688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:45.077193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:45.084519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 178/358] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:44 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:45.089504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:45.141639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:45.193132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:45.265266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55054843-a5b1-4dda-85f6-b14a230b2e80 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:45.289861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67eae29-3b9c-40a7-97cd-c356e107d146 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:45.330353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:45.331036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.331248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.331515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.346207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebc0d9-bedb-4489-a29f-d489a885a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:45.347446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebc0d9-bedb-4489-a29f-d489a885a0eb None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.347987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebc0d9-bedb-4489-a29f-d489a885a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.348394 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebc0d9-bedb-4489-a29f-d489a885a0eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.367307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebc0d9-bedb-4489-a29f-d489a885a0eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-0cebc0d9-bedb-4489-a29f-d489a885a0eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:45.374806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:45.375881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 15:45:45.376795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:45.380334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 181/359] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:45.382006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbbbd4-d608-4bf4-8eba-d8cbb4cf0ab2 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.382428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbbbd4-d608-4bf4-8eba-d8cbb4cf0ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.382849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbbbd4-d608-4bf4-8eba-d8cbb4cf0ab2 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.386357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 182/360] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:45.387574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbbbd4-d608-4bf4-8eba-d8cbb4cf0ab2 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:45.428648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:45.430555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:45.465231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:45.574150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 179/361] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:45.576028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:45.641161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbbbd4-d608-4bf4-8eba-d8cbb4cf0ab2 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:45.716025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530bf64-cca1-46f7-9c5f-b6ff8cda4852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:45.717110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530bf64-cca1-46f7-9c5f-b6ff8cda4852 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.717592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530bf64-cca1-46f7-9c5f-b6ff8cda4852 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.718090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530bf64-cca1-46f7-9c5f-b6ff8cda4852 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.729629 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:45.746516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530bf64-cca1-46f7-9c5f-b6ff8cda4852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-9530bf64-cca1-46f7-9c5f-b6ff8cda4852 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:45.752834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:45.753714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:45.754609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:45.755289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 183/362] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:45.760742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664a35c-f7a3-43b3-8ab5-977f12151f7b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.760742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664a35c-f7a3-43b3-8ab5-977f12151f7b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.760742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664a35c-f7a3-43b3-8ab5-977f12151f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.770692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff17da-670f-4ab4-b882-d9ec9621471c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:45.778697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a664a35c-f7a3-43b3-8ab5-977f12151f7b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:45.779879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a664a35c-f7a3-43b3-8ab5-977f12151f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:45.820370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:45.836183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:45.837729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:45.838012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:45.838283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:45.838957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:45.839259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:45.840657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:45.855610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 180/363] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:45.856453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:45.864115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 184/364] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:45.865082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664a35c-f7a3-43b3-8ab5-977f12151f7b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:45.981350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664a35c-f7a3-43b3-8ab5-977f12151f7b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:45.998212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:46.025216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.026009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.026292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.026559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.027275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:46.027576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:46.028976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:46.038367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 185/365] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:46.039198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:46.116064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62de566-42cc-4294-b0e2-75c7acfb2af9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:46.157383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ad1f3-5afb-4875-99f4-4fb27d5406eb None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:46.196487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.197176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.197508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.197913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] PATH_INFO: `/v3/users/dff9d86ed8b047ee85ff610e17d01a7f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.199305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] RBAC: Authorizing `identity:get_user(user_id=dff9d86ed8b047ee85ff610e17d01a7f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:46.200886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:46.202591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 186/366] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/users/dff9d86ed8b047ee85ff610e17d01a7f => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:46.204352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:46.228657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.229310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.229590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.229862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.231734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:46.238152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:46.239933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 181/367] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:45 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:46.240709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:46.263126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:46.347485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10fa3f-22ce-4121-b8bb-4bbca66808bc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:46.370080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:46.390049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.390570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.390845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.391114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.445200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5272-f40f-4dbf-b360-dd3a3ec7a33d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:46.496386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:46.498221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:46.504664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.505545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.505996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.506419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.507294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] RBAC: Adding query filter params (role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f, user.id=dff9d86ed8b047ee85ff610e17d01a7f, scope.system=all) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:46.507759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:46.509433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:46.524544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 182/368] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/role_assignments?user.id=dff9d86ed8b047ee85ff610e17d01a7f&scope.system=all&role.id=5182eb7e9f3a4b9b8d563e8bf9888d0f => generated 510 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:46.525817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:46.549846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:46.662104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 187/369] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:46.662927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:46.672595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70da6e5-9698-464e-96e0-318a325f3cb1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:46.690478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:46.719451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.720102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.721201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.721479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.723489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:46.726734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:46.728440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 183/370] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:46.733036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:46.840259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffbf76-1d76-41a1-bc2b-f4dd3729b21f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:46.877256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff79dc0-b064-483c-951f-7ba59937829c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:46.897598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.898602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.899110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.899581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.914596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:46.920490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:46.921580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:46.922389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:46.924473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 188/371] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:46.925562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:46.930942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:46.932027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:46.932832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:46.933202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:46.933705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:46.934968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:46.938297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:46.939151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 184/372] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:46.940055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:47.002212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:47.045623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f6280-665c-4138-81c5-e09f20f04d6e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:47.057433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a21fed-7406-4a6c-9494-f56a93361c88 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:47.090780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:47.091933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.092394 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.094220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.095519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:47.096076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:47.099272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:47.100465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:47.101262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.101725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.102010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.102781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:47.103538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:47.104703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:47.106944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 189/373] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:47.108258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:47.123148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 185/374] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:47.128955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:47.240949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:47.276093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ed83-663e-4487-b7a1-ea602ad33347 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:47.293515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90272c8e-fca8-462d-a0ae-75a5b9046073 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:47.317303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:47.337307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13117cab-bb4c-49da-bc08-94ea6682be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:47.338238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13117cab-bb4c-49da-bc08-94ea6682be4a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.338674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13117cab-bb4c-49da-bc08-94ea6682be4a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.339075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13117cab-bb4c-49da-bc08-94ea6682be4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.342619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:47.343560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.344029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.344475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.346479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:47.348268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:47.355827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 190/375] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:47 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:47.357070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:47.420974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13117cab-bb4c-49da-bc08-94ea6682be4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:47.422583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13117cab-bb4c-49da-bc08-94ea6682be4a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:47.534665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 186/376] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 15:45:47.534947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1d092-9225-470e-b972-b136bdce0048 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:47.535908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519e51f-c306-4646-8aff-3a051df2c121 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.536189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519e51f-c306-4646-8aff-3a051df2c121 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.536462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519e51f-c306-4646-8aff-3a051df2c121 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.537954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 187/377] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:47.543033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ca61-0f92-4f25-89ea-e34bf81f47af None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.543313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ca61-0f92-4f25-89ea-e34bf81f47af None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.543582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ca61-0f92-4f25-89ea-e34bf81f47af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.562297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-0ab0ca61-0f92-4f25-89ea-e34bf81f47af None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:47.563481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0ab0ca61-0f92-4f25-89ea-e34bf81f47af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:47.575102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:47.584643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:47.586081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.586575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.587027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.587939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:47.589021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:47.590958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:47.597147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:47.602313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 191/378] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:47 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:47.603770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:47.664672 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 188/379] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:47.752681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46e0a6-74df-489f-a3fd-322e0632ba65 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:47.785794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680393-91aa-40ed-8718-be955995bd86 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.786085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680393-91aa-40ed-8718-be955995bd86 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.786432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680393-91aa-40ed-8718-be955995bd86 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.788254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 189/380] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:47.794057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4517-96c8-46cf-8a2f-07a1380ee573 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.794789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4517-96c8-46cf-8a2f-07a1380ee573 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.796413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c4517-96c8-46cf-8a2f-07a1380ee573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.809295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:47.810338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:47.811676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:47.812140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] PATH_INFO: `/v3/projects/0727be3aa6194ac280b9115cbd69b50d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:47.819480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] RBAC: Authorizing `identity:get_project(project_id=0727be3aa6194ac280b9115cbd69b50d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:47.821423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:47.823382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 192/381] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:47 2022] GET /identity/v3/projects/0727be3aa6194ac280b9115cbd69b50d => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:47.829232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:47.834054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5c9c4517-96c8-46cf-8a2f-07a1380ee573 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:47.836448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5c9c4517-96c8-46cf-8a2f-07a1380ee573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:47.958342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:47.973813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 190/382] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:48.000353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20be64-1a54-447c-84b2-7c1b598909d8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:48.036441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:48.037074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:48.037355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:48.037626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] PATH_INFO: `/v3/groups/182654cf55054ab9bd8a21aa6f923f2e` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:48.044001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] RBAC: Authorizing `identity:get_group(group_id=182654cf55054ab9bd8a21aa6f923f2e)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:48.045596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:48.047538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 193/383] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:47 2022] GET /identity/v3/groups/182654cf55054ab9bd8a21aa6f923f2e => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:48.054121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4517-96c8-46cf-8a2f-07a1380ee573 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:48.074296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:48.192158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:48.254157 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721651a-8a3b-45a2-a21e-d1764657a046 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:48.269223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c4517-96c8-46cf-8a2f-07a1380ee573 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:48.286887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:48.293210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:48.294118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:48.294591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:48.295072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:48.320859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:48.326364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:48.327244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:48.328019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:48.330332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 194/384] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:48.331453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:48.343291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:48.343910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:48.344189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:48.344464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:48.345718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] RBAC: Adding query filter params (group.id=182654cf55054ab9bd8a21aa6f923f2e, role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=0727be3aa6194ac280b9115cbd69b50d) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:48.346036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:48.349300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:48.359060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 191/385] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/role_assignments?group.id=182654cf55054ab9bd8a21aa6f923f2e&scope.project.id=0727be3aa6194ac280b9115cbd69b50d&role.id=0cb19d985d3d456983e0b1624610b058 => generated 260 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:48.359060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:48.523845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a7ce8-fd92-400b-a6f3-c6e6a0fb11e9 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:48.535021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee3bde-fecd-4aae-81e9-08cdf28a1844 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:48.570627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:48.571392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:48.571677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:48.571948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:48.584712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:48.590769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:48.591639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:48.593903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 195/386] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:48.598257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:48.609456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:48.610086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:48.610367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:48.610649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:48.611340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:48.611647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:48.613178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:48.628010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:48.635218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 192/387] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:48.639603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:48.678900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:48.758527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acb5a6-3124-4072-ae68-7b1b8043b338 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:48.813762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:48.814514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:48.814795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:48.815061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:48.815724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:48.816020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:48.819638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:48.831645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 196/388] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:48.836928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:48.839208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011260cb-3b85-42d5-b56d-010dc5f173c1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:48.925587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:48.927061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:48.927330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:48.927594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:48.928825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:48.930342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:48.932948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 193/389] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:48.937411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:49.014736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459c31f-1c76-41d7-8886-330e09d265ea None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:49.039781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:49.053510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:49.079034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:49.079677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:49.079949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:49.080211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:49.082668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:49.086441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:49.088046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 197/390] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:49.094657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:49.118132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422d6cc-426c-47dc-81d8-b3c9a3fc87ae None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:49.181875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:49.183642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:49.183918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:49.184199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:49.188789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:49.190262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:49.193911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 194/391] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:48 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:49.199347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:49.274146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c7156-cbae-4793-881a-227b3b61df40 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:49.329096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:49.329771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:49.330049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:49.330321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:49.334182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:49.335927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:49.337968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 198/392] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:49 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:49.344228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:49.420473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:49.452524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f70b9f-a53c-4a0d-a922-f97bf0129f42 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:49.474442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd97170-8357-4402-bdc7-10113d841ec6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:49.521521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd44fc4-2849-45bf-9eea-fd695cf3fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:49.522201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd44fc4-2849-45bf-9eea-fd695cf3fe53 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:49.522478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd44fc4-2849-45bf-9eea-fd695cf3fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:49.522752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd44fc4-2849-45bf-9eea-fd695cf3fe53 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74/groups/fa3e36820d3f46b08db95628284c364d/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:49.525623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc523d3-7f5a-4e81-b174-512620a14cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:49.526287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc523d3-7f5a-4e81-b174-512620a14cd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:49.526563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc523d3-7f5a-4e81-b174-512620a14cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:49.526832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc523d3-7f5a-4e81-b174-512620a14cd8 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74/groups/fa3e36820d3f46b08db95628284c364d/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:49.528251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd44fc4-2849-45bf-9eea-fd695cf3fe53 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7ac2137e95a4380a28ffb6603712f74, group_id=fa3e36820d3f46b08db95628284c364d, role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:49.530008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc523d3-7f5a-4e81-b174-512620a14cd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7ac2137e95a4380a28ffb6603712f74, group_id=fa3e36820d3f46b08db95628284c364d, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:49.531476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc523d3-7f5a-4e81-b174-512620a14cd8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:49.534170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd44fc4-2849-45bf-9eea-fd695cf3fe53 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:49.540578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:49.563087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 199/393] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 15:45:49 2022] PUT /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74/groups/fa3e36820d3f46b08db95628284c364d/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:49.564905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 195/394] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 15:45:49 2022] PUT /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74/groups/fa3e36820d3f46b08db95628284c364d/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:45:50.011240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db149-efb5-41f4-942b-9fb77fd329d5 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:50.011769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db149-efb5-41f4-942b-9fb77fd329d5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:50.012175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db149-efb5-41f4-942b-9fb77fd329d5 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:50.013982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 200/395] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:50.019728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44754e51-8e2b-4a4f-ab85-ad02c74b40e4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:50.020285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44754e51-8e2b-4a4f-ab85-ad02c74b40e4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:50.020800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44754e51-8e2b-4a4f-ab85-ad02c74b40e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:50.041052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-44754e51-8e2b-4a4f-ab85-ad02c74b40e4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:50.042577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-44754e51-8e2b-4a4f-ab85-ad02c74b40e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:50.139382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 196/396] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:50.435691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e1290-c26c-4bc2-938b-23e9592efce9 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:50.436375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e1290-c26c-4bc2-938b-23e9592efce9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:50.436895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e1290-c26c-4bc2-938b-23e9592efce9 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:50.438425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 201/397] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:50.443777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353cd8d-7461-4884-8a31-ad6b13df7f76 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:50.444048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353cd8d-7461-4884-8a31-ad6b13df7f76 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:50.444325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353cd8d-7461-4884-8a31-ad6b13df7f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:50.469782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d353cd8d-7461-4884-8a31-ad6b13df7f76 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:50.470982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d353cd8d-7461-4884-8a31-ad6b13df7f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:50.553386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e1290-c26c-4bc2-938b-23e9592efce9 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:50.562173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 197/398] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:50.710057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e1290-c26c-4bc2-938b-23e9592efce9 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:50.725635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:50.754740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:50.755411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:50.755654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:50.755923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:50.766760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:50.771819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:50.772689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:50.773659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 202/399] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:50.780331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353cd8d-7461-4884-8a31-ad6b13df7f76 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:50.873781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:50.937993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353cd8d-7461-4884-8a31-ad6b13df7f76 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:50.977082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:50.977612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:50.977886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:50.978153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:50.978821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:50.979119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:50.980503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:50.989933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 198/400] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:50.996331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.031445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.187930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa63f5-8a36-4df9-a63b-c2b1f2b46280 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:51.190149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce23444-9b5e-48b9-8469-5d3a3a639a3a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:51.230067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:51.236097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:51.236748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.237146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.239581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.241754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:51.243209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:51.244795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 203/401] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:50 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:51.250357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.267252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc72b43-9d39-4648-ab5d-c5a4ac09a765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:51.267897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc72b43-9d39-4648-ab5d-c5a4ac09a765 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.268175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc72b43-9d39-4648-ab5d-c5a4ac09a765 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.268449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc72b43-9d39-4648-ab5d-c5a4ac09a765 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.282764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc72b43-9d39-4648-ab5d-c5a4ac09a765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-dbc72b43-9d39-4648-ab5d-c5a4ac09a765 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:51.290842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:51.291851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:51.292632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:51.293259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 199/402] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:51.293993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b752495-da29-464a-8877-b6f125286a00 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.294276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b752495-da29-464a-8877-b6f125286a00 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.294548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b752495-da29-464a-8877-b6f125286a00 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.304814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 200/403] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:51.305798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b752495-da29-464a-8877-b6f125286a00 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.313068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:51.390095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69f717-fddf-4eaa-8154-abc1cd5616cd None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:51.439303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f836e-50d3-483f-bc12-684e8b4749e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:51.439940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f836e-50d3-483f-bc12-684e8b4749e7 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.440200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f836e-50d3-483f-bc12-684e8b4749e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.440480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f836e-50d3-483f-bc12-684e8b4749e7 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.441752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f836e-50d3-483f-bc12-684e8b4749e7 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:51.443401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f836e-50d3-483f-bc12-684e8b4749e7 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:51.445422 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 204/404] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:51 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:51.446630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183404f-e56d-45b9-9e3e-2aa16e960212 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.446910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183404f-e56d-45b9-9e3e-2aa16e960212 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.447176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183404f-e56d-45b9-9e3e-2aa16e960212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.453556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b752495-da29-464a-8877-b6f125286a00 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:51.465471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-1183404f-e56d-45b9-9e3e-2aa16e960212 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:51.466655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1183404f-e56d-45b9-9e3e-2aa16e960212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:51.499099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:51.499811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.500095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.500451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.501158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:51.501577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:51.503154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:51.513720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 201/405] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:51.514656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.547613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 205/406] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:51.548521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183404f-e56d-45b9-9e3e-2aa16e960212 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.569414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:51.597136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:51.668999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa59d8-b4c7-43c2-aee5-c97768e7e9fa None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:51.670231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1183404f-e56d-45b9-9e3e-2aa16e960212 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:51.728819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:51.730260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.730824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.731686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.734229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:51.736254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:51.739557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:51.740392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.740828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.741233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/groups/fa3e36820d3f46b08db95628284c364d/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.743024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 206/407] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:51 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:51.744421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a0f353314fa4f1f8740e6d18f9374c6, group_id=fa3e36820d3f46b08db95628284c364d, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:51.746036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:51.748851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.774493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 202/408] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 15:45:51 2022] PUT /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/groups/fa3e36820d3f46b08db95628284c364d/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:51.885375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6afd4c-5f44-407a-b8b1-7220638576c0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:51.888156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:51.904357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:51.949066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:51.950028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:51.950596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:51.951075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:51.953466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:51.955491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:51.958208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 207/409] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:51 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:51.968559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:52.077741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0cbb4e-44a6-4c2f-bade-2541cdae8216 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:52.089645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:52.132758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b099b39-0ccb-43c0-a580-29e84f5a28aa None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:52.137469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:52.138123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.138442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.138711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.158593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:52.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:52.164358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:52.165385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 203/410] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:52.171395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:52.209265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:52.210762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.211043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.211318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/groups/fa3e36820d3f46b08db95628284c364d/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.216338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] RBAC: Authorizing `identity:create_grant(project_id=8a0f353314fa4f1f8740e6d18f9374c6, group_id=fa3e36820d3f46b08db95628284c364d, role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:52.218879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:52.243692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 208/411] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 15:45:51 2022] PUT /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6/groups/fa3e36820d3f46b08db95628284c364d/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:45:52.362010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213603c6-3175-4c74-a4cf-0ba605cd3c0a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:52.418648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b519f-4259-43ab-a12d-f45c7ae5930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:52.419267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b519f-4259-43ab-a12d-f45c7ae5930f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.419539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b519f-4259-43ab-a12d-f45c7ae5930f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.419803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b519f-4259-43ab-a12d-f45c7ae5930f None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.420550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b519f-4259-43ab-a12d-f45c7ae5930f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:52.420878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b519f-4259-43ab-a12d-f45c7ae5930f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:52.422362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7b519f-4259-43ab-a12d-f45c7ae5930f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:52.434209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 204/412] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:52.439823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:52.485414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:52.508469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f11f9b-de4a-4509-8324-86b1b16296c9 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.508941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f11f9b-de4a-4509-8324-86b1b16296c9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.509315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f11f9b-de4a-4509-8324-86b1b16296c9 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.510839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 205/413] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:52.516581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c685d6-ffe6-4363-87ed-0c47311f9d85 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.516992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c685d6-ffe6-4363-87ed-0c47311f9d85 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.517365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c685d6-ffe6-4363-87ed-0c47311f9d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.538604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-91c685d6-ffe6-4363-87ed-0c47311f9d85 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:52.540017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-91c685d6-ffe6-4363-87ed-0c47311f9d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:52.562386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:52.568534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804855dc-dba9-4cd9-87b4-25f7641e894d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:52.615568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ba287-9435-4b3c-b59f-f7113f84a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:52.617438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ba287-9435-4b3c-b59f-f7113f84a2cc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.617964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ba287-9435-4b3c-b59f-f7113f84a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.618378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ba287-9435-4b3c-b59f-f7113f84a2cc None admin] PATH_INFO: `/v3/groups/182654cf55054ab9bd8a21aa6f923f2e` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.620361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ba287-9435-4b3c-b59f-f7113f84a2cc None admin] RBAC: Authorizing `identity:get_group(group_id=182654cf55054ab9bd8a21aa6f923f2e)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:52.622563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ba287-9435-4b3c-b59f-f7113f84a2cc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:52.625330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 209/414] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:52 2022] GET /identity/v3/groups/182654cf55054ab9bd8a21aa6f923f2e => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:52.626807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887561-a36a-4ad0-8acd-4999529ad336 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.627486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887561-a36a-4ad0-8acd-4999529ad336 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.628127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1887561-a36a-4ad0-8acd-4999529ad336 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.628250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 206/415] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:52.637173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c685d6-ffe6-4363-87ed-0c47311f9d85 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:52.637313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 210/416] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:52.643258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296121-ae0a-4823-bb90-06b1e4855411 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.643538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296121-ae0a-4823-bb90-06b1e4855411 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.645439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f296121-ae0a-4823-bb90-06b1e4855411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.678443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-3f296121-ae0a-4823-bb90-06b1e4855411 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:52.679632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3f296121-ae0a-4823-bb90-06b1e4855411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:52.799782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c685d6-ffe6-4363-87ed-0c47311f9d85 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:52.819005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 211/417] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:52.856787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:52.857423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:52.857693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:52.857956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] PATH_INFO: `/v3/projects/0727be3aa6194ac280b9115cbd69b50d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:52.861401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] RBAC: Authorizing `identity:get_project(project_id=0727be3aa6194ac280b9115cbd69b50d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:52.862851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:52.865335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 207/418] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:52 2022] GET /identity/v3/projects/0727be3aa6194ac280b9115cbd69b50d => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:52.871258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f296121-ae0a-4823-bb90-06b1e4855411 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:52.902021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:52.941071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:52.974779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:52.975765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f296121-ae0a-4823-bb90-06b1e4855411 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:53.021461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:53.022408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:53.022408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:53.022686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] PATH_INFO: `/v3/projects/0727be3aa6194ac280b9115cbd69b50d/groups/182654cf55054ab9bd8a21aa6f923f2e/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:53.024660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] RBAC: Authorizing `identity:create_grant(project_id=0727be3aa6194ac280b9115cbd69b50d, group_id=182654cf55054ab9bd8a21aa6f923f2e, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:53.026167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:53.051010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 212/419] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Oct 3 15:45:52 2022] PUT /identity/v3/projects/0727be3aa6194ac280b9115cbd69b50d/groups/182654cf55054ab9bd8a21aa6f923f2e/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:45:53.069495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4233cf-2f9c-4f96-aa54-bb1794250949 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:53.108983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:53.110166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:53.110644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:53.111114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:53.123170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:53.128435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:53.130024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:53.130829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:53.133385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 208/420] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:53.139090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:53.139237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:53.237522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:53.260839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe284ac7-ee5c-4295-ac4e-edb28a6adbe3 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:53.301224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:53.301847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:53.302121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:53.302390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:53.303167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:53.303482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:53.304939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:53.315970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 209/421] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:53.321822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:53.340797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:53.363882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a68a8-73eb-4212-bc22-883d1a891a39 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:53.402411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:53.403229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:53.403504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:53.403772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:53.418505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:53.426565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:53.427480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:53.428246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:53.432500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 213/422] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:53.438747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:53.487866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624cf3-6b0b-48d2-9488-26c481330067 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:53.575461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:53.576073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:53.576582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:53.576881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:53.605387 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:53.607636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16f08c-a4e0-4e97-b05a-ed072ace1a98 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:53.679421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:53.680019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:53.680279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:53.680555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:53.681300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] RBAC: Adding query filter params (name=member) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:53.681604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:53.687294 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:53.699954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 214/423] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:53.706035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:53.721725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:53.724439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:53.725042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:53.847036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3d5c-251b-41ec-b1a0-c886bdc34022 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:53.912805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:53.913438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:53.913854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:53.914150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:53.937953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 210/424] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:53.945081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:54.013906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:54.040084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:54.041784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:54.065067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:54.133391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6e637-3537-4295-b2fe-cd6253ca46dc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:54.197856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:54.198412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.198685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.198960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.200219 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:54.201716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:54.209881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 211/425] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:53 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:54.213185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:54.231940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 215/426] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:54.237159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:54.364584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e637c-ad05-4488-af83-1a860ea675d1 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:54.379519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:54.420121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:54.420802 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.421085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.421359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] PATH_INFO: `/v3/projects/d7ac2137e95a4380a28ffb6603712f74` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.422741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=d7ac2137e95a4380a28ffb6603712f74)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:54.424205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:54.428905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 216/427] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:54 2022] GET /identity/v3/projects/d7ac2137e95a4380a28ffb6603712f74 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:54.434141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af104-8da2-4315-a0c9-cf02b2e6c991 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:54.435352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:54.453449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:54.498578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:54.499224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.499501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.499772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.501515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:54.502987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:54.506127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 212/428] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:54 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:54.516564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:54.595220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324de1-e520-4e82-a0c3-25f9e9608ff4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:54.630148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e60f66-579d-484e-a651-3da1627b232f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:54.636472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d721f59-f0c6-496e-9ecd-6efc1e5d5eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:54.637183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721f59-f0c6-496e-9ecd-6efc1e5d5eaf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.637486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721f59-f0c6-496e-9ecd-6efc1e5d5eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.637826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d721f59-f0c6-496e-9ecd-6efc1e5d5eaf None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.638943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721f59-f0c6-496e-9ecd-6efc1e5d5eaf None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:54.640582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d721f59-f0c6-496e-9ecd-6efc1e5d5eaf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:54.642663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 217/429] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:54 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:54.645192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170d7ac-0f3b-4f6e-b406-dd651de0ebe7 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.645569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170d7ac-0f3b-4f6e-b406-dd651de0ebe7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.645850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170d7ac-0f3b-4f6e-b406-dd651de0ebe7 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.647335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 218/430] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:54.652899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170d7ac-0f3b-4f6e-b406-dd651de0ebe7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:54.676029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b9c10-0b15-4a7b-82cb-69f31942c732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:54.676901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b9c10-0b15-4a7b-82cb-69f31942c732 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.677183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b9c10-0b15-4a7b-82cb-69f31942c732 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.677454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b9c10-0b15-4a7b-82cb-69f31942c732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.678194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b9c10-0b15-4a7b-82cb-69f31942c732 None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=d7ac2137e95a4380a28ffb6603712f74) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:54.678500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b9c10-0b15-4a7b-82cb-69f31942c732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:54.680133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b9c10-0b15-4a7b-82cb-69f31942c732 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:54.692840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 213/431] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:54 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:54.694446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bded6-2fc4-4655-b63d-0fa91eb71b8e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.695038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bded6-2fc4-4655-b63d-0fa91eb71b8e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.695317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bded6-2fc4-4655-b63d-0fa91eb71b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.716855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:54.742079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2b6bded6-2fc4-4655-b63d-0fa91eb71b8e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:54.743432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2b6bded6-2fc4-4655-b63d-0fa91eb71b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:54.781839 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170d7ac-0f3b-4f6e-b406-dd651de0ebe7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:54.822714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8e1d6-9267-493e-b9d0-45060dc9acda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:54.823682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e1d6-9267-493e-b9d0-45060dc9acda None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.824376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e1d6-9267-493e-b9d0-45060dc9acda None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.824991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8e1d6-9267-493e-b9d0-45060dc9acda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.825927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8e1d6-9267-493e-b9d0-45060dc9acda None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=d7ac2137e95a4380a28ffb6603712f74) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:54.826401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8e1d6-9267-493e-b9d0-45060dc9acda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:54.828084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8e1d6-9267-493e-b9d0-45060dc9acda None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:54.833104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:54.836353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 219/432] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:54 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=d7ac2137e95a4380a28ffb6603712f74&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:54.885610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052869e8-2c6e-4a4a-97a6-acbd31cc2c6a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.885895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052869e8-2c6e-4a4a-97a6-acbd31cc2c6a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.886167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052869e8-2c6e-4a4a-97a6-acbd31cc2c6a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.887744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 214/434] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:54.887865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 220/434] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:45:54.893332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf11c89-3b42-4fcc-af94-4d460234eae9 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:54.893846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf11c89-3b42-4fcc-af94-4d460234eae9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:54.894942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf11c89-3b42-4fcc-af94-4d460234eae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:54.916963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bdf11c89-3b42-4fcc-af94-4d460234eae9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:54.918438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bdf11c89-3b42-4fcc-af94-4d460234eae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:55.013179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 221/435] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:45:55.230098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bded6-2fc4-4655-b63d-0fa91eb71b8e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:55.303525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf11c89-3b42-4fcc-af94-4d460234eae9 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:55.345301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bded6-2fc4-4655-b63d-0fa91eb71b8e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:55.359947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:55.385603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:55.386246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.386525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.386796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.402468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:55.407598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:55.408558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 15:45:55.409421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:45:55.412313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 215/436] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:55.418204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:55.433963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf11c89-3b42-4fcc-af94-4d460234eae9 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:55.453741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:55.480218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:55.481045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.481382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.481713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.495763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:55.500605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:55.501503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:45:55.502283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:55.507982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 222/437] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:45:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:45:55.512160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:55.552212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e45688-e5d6-46da-a058-f60069863530 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:55.606308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:55.606975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.607256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.607532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.608230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] RBAC: Adding query filter params (name=member) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:55.608539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:55.610108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:55.624776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 216/438] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:45:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:55.626820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:55.664144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a59b4-c837-4cea-a761-895df0876fc1 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:55.666670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:55.701416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c412fb-2704-4f05-afea-91e6e953aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:55.702018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c412fb-2704-4f05-afea-91e6e953aa5a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.702291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c412fb-2704-4f05-afea-91e6e953aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.702555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c412fb-2704-4f05-afea-91e6e953aa5a None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.703916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c412fb-2704-4f05-afea-91e6e953aa5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:55.703916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c412fb-2704-4f05-afea-91e6e953aa5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:55.704901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c412fb-2704-4f05-afea-91e6e953aa5a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:55.713896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 223/439] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:45:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:55.715182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f74f-c620-492a-bed8-f33d2d2ebf33 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.715466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f74f-c620-492a-bed8-f33d2d2ebf33 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.715733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f74f-c620-492a-bed8-f33d2d2ebf33 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.716926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 224/440] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:45:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:45:55.719379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740f74f-c620-492a-bed8-f33d2d2ebf33 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:55.732542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:55.740104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f32cac-fa05-4c3b-ba1c-32c94a9bd39c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:55.781300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43894f-8f16-41e5-876b-fec9cfaf7851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:55.781932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43894f-8f16-41e5-876b-fec9cfaf7851 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.782168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43894f-8f16-41e5-876b-fec9cfaf7851 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.782433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43894f-8f16-41e5-876b-fec9cfaf7851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.821749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740f74f-c620-492a-bed8-f33d2d2ebf33 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:55.849503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43894f-8f16-41e5-876b-fec9cfaf7851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:55.850977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43894f-8f16-41e5-876b-fec9cfaf7851 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:55.856922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:55.857467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.857746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.858016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.888332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:55.925327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:55.926840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:55.942425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:55.950630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 217/441] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 15:45:55.955378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612623c-fd57-4105-9d85-e46326a0accc None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:55.955378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612623c-fd57-4105-9d85-e46326a0accc None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:55.955378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612623c-fd57-4105-9d85-e46326a0accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:55.969731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-4612623c-fd57-4105-9d85-e46326a0accc None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:45:55.970933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-4612623c-fd57-4105-9d85-e46326a0accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:45:56.026437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 225/442] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 15:45:56.027277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.041306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 218/443] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:45:56.042381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612623c-fd57-4105-9d85-e46326a0accc None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.095432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:56.136378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202da79-6d55-49ca-a412-971d759b1f8f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.154205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612623c-fd57-4105-9d85-e46326a0accc None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.173257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.173890 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.174167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.174863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.176063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.177543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:56.178358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:56.179199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 226/444] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:56.185024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.190155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.190780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.191594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.191858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] PATH_INFO: `/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.193016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] RBAC: Authorizing `identity:get_project(project_id=8a0f353314fa4f1f8740e6d18f9374c6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.194461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:56.196126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 219/445] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/projects/8a0f353314fa4f1f8740e6d18f9374c6 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:56.201610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.297722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701c507-0fdf-44de-bf08-1a600d24a187 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.312508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d3ad6-1f73-46a6-81e1-476961b162aa None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.337214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:56.342783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.343404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.343675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.343940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.345084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.346529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:56.348390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.349175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 227/446] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:56.349311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.349562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.349827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] PATH_INFO: `/v3/groups/fa3e36820d3f46b08db95628284c364d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.349921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.350962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] RBAC: Authorizing `identity:get_group(group_id=fa3e36820d3f46b08db95628284c364d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.352602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:56.355127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 220/447] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/groups/fa3e36820d3f46b08db95628284c364d => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:56.355783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.414946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:56.450623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eb148-1c88-4525-81e9-5c98daed937c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.457336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd466d5-8a9f-4245-8d80-5ceaa0223f5c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.487016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.487644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.488506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.488812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.490844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.491445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.491718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.491985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.492706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=5a57b9470d8a4718a12b9e65777a95df, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:56.493031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.494489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:56.499000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:45:56.500131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 221/448] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=5a57b9470d8a4718a12b9e65777a95df => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:56.500925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:45:56.505189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:45:56.506388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:45:56.507165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 228/449] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:45:56.512428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.584492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:56.625152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d82ff4-096d-4122-b787-38c217d80f22 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.633003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400a2cd-78dc-4da1-aa7c-a9ae78e0493c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.652647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:56.665994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.666536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.666797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.667071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.667808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] RBAC: Adding query filter params (group.id=fa3e36820d3f46b08db95628284c364d, role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:56.668119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.669624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:56.671599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.672204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.672474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.673078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.673510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:56.673791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.675179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:56.676659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 222/450] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/role_assignments?group.id=fa3e36820d3f46b08db95628284c364d&scope.project.id=8a0f353314fa4f1f8740e6d18f9374c6&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:56.689515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 229/451] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:45:56.691621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:56.813045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbef4-b5c9-47fc-b28d-6c236e821c6e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:56.851717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:56.852334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:56.852614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:56.852916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:56.911111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:56.924983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:56.926851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:57.027278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 223/452] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:45:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:45:57.033243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:57.131597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610a8d1-7cd8-4459-9d9b-42422ffbd622 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:57.152728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:57.164593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:57.165236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:57.165512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:57.165777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] PATH_INFO: `/v3/projects/0727be3aa6194ac280b9115cbd69b50d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:57.166873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] RBAC: Authorizing `identity:get_project(project_id=0727be3aa6194ac280b9115cbd69b50d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:57.168296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:57.169927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 230/453] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:45:57 2022] GET /identity/v3/projects/0727be3aa6194ac280b9115cbd69b50d => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:57.175680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:57.260612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:45:57.275355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66a0e1-5d07-4799-a23a-2f9146bacbcc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:57.309056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f401e-81f7-4de1-bebc-8b69bc87498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:57.309815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f401e-81f7-4de1-bebc-8b69bc87498f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:57.310322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f401e-81f7-4de1-bebc-8b69bc87498f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:57.310827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f401e-81f7-4de1-bebc-8b69bc87498f None admin] PATH_INFO: `/v3/groups/182654cf55054ab9bd8a21aa6f923f2e` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:57.312821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f401e-81f7-4de1-bebc-8b69bc87498f None admin] RBAC: Authorizing `identity:get_group(group_id=182654cf55054ab9bd8a21aa6f923f2e)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:57.315562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f401e-81f7-4de1-bebc-8b69bc87498f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:57.318522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 224/454] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Oct 3 15:45:57 2022] GET /identity/v3/groups/182654cf55054ab9bd8a21aa6f923f2e => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:45:57.324323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:45:57.431691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c90f2-6f2e-416f-987a-b54d7151c81e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:45:57.464510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1179be7-a5b5-43de-a58e-2f04d07acfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:45:57.465124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1179be7-a5b5-43de-a58e-2f04d07acfab None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:45:57.465404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1179be7-a5b5-43de-a58e-2f04d07acfab None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:45:57.465672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1179be7-a5b5-43de-a58e-2f04d07acfab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:45:57.466393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1179be7-a5b5-43de-a58e-2f04d07acfab None admin] RBAC: Adding query filter params (group.id=182654cf55054ab9bd8a21aa6f923f2e, role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=0727be3aa6194ac280b9115cbd69b50d) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:45:57.466699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1179be7-a5b5-43de-a58e-2f04d07acfab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:45:57.468125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1179be7-a5b5-43de-a58e-2f04d07acfab None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:45:57.474336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 231/455] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Oct 3 15:45:57 2022] GET /identity/v3/role_assignments?group.id=182654cf55054ab9bd8a21aa6f923f2e&scope.project.id=0727be3aa6194ac280b9115cbd69b50d&role.id=0cb19d985d3d456983e0b1624610b058 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:00.018228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999720fb-b89e-4f8b-bc4d-ee27a3a93b77 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.020106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999720fb-b89e-4f8b-bc4d-ee27a3a93b77 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.020637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999720fb-b89e-4f8b-bc4d-ee27a3a93b77 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.022629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 225/456] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:00.028677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe7e32-057d-4581-abed-8842369fcaad None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.029532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe7e32-057d-4581-abed-8842369fcaad None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.030020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe7e32-057d-4581-abed-8842369fcaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.049233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-44fe7e32-057d-4581-abed-8842369fcaad None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:00.051433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-44fe7e32-057d-4581-abed-8842369fcaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:00.115011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:00.133468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 232/457] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:00.190169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82c2a5-7cbf-45d4-8e9b-260fe4d11b8d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.190819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82c2a5-7cbf-45d4-8e9b-260fe4d11b8d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.191438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82c2a5-7cbf-45d4-8e9b-260fe4d11b8d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.193017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 226/458] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:00.198615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed161a73-8e1f-4ac6-a186-a4465606c96b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.198899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed161a73-8e1f-4ac6-a186-a4465606c96b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.199168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed161a73-8e1f-4ac6-a186-a4465606c96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.220512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ed161a73-8e1f-4ac6-a186-a4465606c96b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:00.221714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ed161a73-8e1f-4ac6-a186-a4465606c96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:00.294833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 233/459] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:00.457301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82c2a5-7cbf-45d4-8e9b-260fe4d11b8d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:00.591492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82c2a5-7cbf-45d4-8e9b-260fe4d11b8d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:00.613641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb6f34-811f-483a-a89e-1fedcf2a3655 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.614205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb6f34-811f-483a-a89e-1fedcf2a3655 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.614659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb6f34-811f-483a-a89e-1fedcf2a3655 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.616418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 234/460] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:00.621736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0da5-ee3a-41b7-a838-9d9d959901c9 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.622013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0da5-ee3a-41b7-a838-9d9d959901c9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.622278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f0da5-ee3a-41b7-a838-9d9d959901c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.649760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-2b9f0da5-ee3a-41b7-a838-9d9d959901c9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:00.650936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2b9f0da5-ee3a-41b7-a838-9d9d959901c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:00.653931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:00.654555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.654825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.655088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.666113 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:00.670495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:00.695442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:00.696624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:00.697562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:00.697562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:00.697562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:00.697562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 227/461] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:00.697801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:00.719660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:00.748318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 235/462] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:00.757140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0da5-ee3a-41b7-a838-9d9d959901c9 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:00.827771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b459f2-9704-48ca-8222-85755af24c61 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:00.841091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:00.870017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:00.871081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:00.871503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:00.871918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:00.884593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:00.886512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:00.902351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:00.903251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:00.904157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:00.904157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:00.904157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:00.906285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 228/463] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:00.919362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:00.942976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f0da5-ee3a-41b7-a838-9d9d959901c9 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:01.006349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:01.007017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.007297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.007568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.008220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:01.008520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:01.010048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:01.028417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 236/464] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:01.033453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:01.034168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:01.121763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14553595-50e7-4d39-a2ec-681cac5a4a39 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:01.161966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:01.162938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.163364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.163799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.164615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:01.165130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:01.166907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:01.181599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 229/465] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:01.186264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:01.203255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c097e9a-c61a-4c2e-9782-adccdd3e72a0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:01.206286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:01.247326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:01.247940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.248803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.249078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.249722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:01.251524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:01.277209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 237/466] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:46:01 2022] POST /identity/v3/users => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:01.277901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:01.304056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b762f-89bb-40f2-92f9-dadcf9c12e85 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:01.334147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:01.353509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5711af-e559-492b-aa84-bd04aa080e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:01.354500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5711af-e559-492b-aa84-bd04aa080e56 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.354962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5711af-e559-492b-aa84-bd04aa080e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.355408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5711af-e559-492b-aa84-bd04aa080e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.367948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5711af-e559-492b-aa84-bd04aa080e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:01.369532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5711af-e559-492b-aa84-bd04aa080e56 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-5e5711af-e559-492b-aa84-bd04aa080e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:01.386850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:01.387840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:01.388671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:01.388671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:01.388671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:01.393116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 230/467] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:01.396860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cc0fa-51c0-499b-b0de-456b432e7558 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.397492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cc0fa-51c0-499b-b0de-456b432e7558 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.397984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cc0fa-51c0-499b-b0de-456b432e7558 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.399740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 231/468] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:01.400467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cc0fa-51c0-499b-b0de-456b432e7558 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:01.426787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53667b06-956f-4d32-930f-4b338d66cc68 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:01.474902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c33f1-75dc-44d5-80ff-904a2c1fae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:01.475946 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c33f1-75dc-44d5-80ff-904a2c1fae88 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.476497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c33f1-75dc-44d5-80ff-904a2c1fae88 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.476979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c33f1-75dc-44d5-80ff-904a2c1fae88 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.477847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c33f1-75dc-44d5-80ff-904a2c1fae88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:01.480278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c33f1-75dc-44d5-80ff-904a2c1fae88 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:01.516217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 238/469] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:46:01 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:01.518005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8908c3-e7df-43b5-acf4-92c08b95785f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.518296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8908c3-e7df-43b5-acf4-92c08b95785f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.518572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8908c3-e7df-43b5-acf4-92c08b95785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.539672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0c8908c3-e7df-43b5-acf4-92c08b95785f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:01.540952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0c8908c3-e7df-43b5-acf4-92c08b95785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:01.543048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cc0fa-51c0-499b-b0de-456b432e7558 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:01.558054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:01.598260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aef6ac-5f58-4b4e-a713-6b77df4c4a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:01.599158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aef6ac-5f58-4b4e-a713-6b77df4c4a63 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.599436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aef6ac-5f58-4b4e-a713-6b77df4c4a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.599702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aef6ac-5f58-4b4e-a713-6b77df4c4a63 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.600359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aef6ac-5f58-4b4e-a713-6b77df4c4a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:01.600657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aef6ac-5f58-4b4e-a713-6b77df4c4a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:01.602126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aef6ac-5f58-4b4e-a713-6b77df4c4a63 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:01.617649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 232/470] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:01.624371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79255d83-e8bd-4ac3-9ce9-5e817f4d2e22 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.624371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79255d83-e8bd-4ac3-9ce9-5e817f4d2e22 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.624371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79255d83-e8bd-4ac3-9ce9-5e817f4d2e22 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.625685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 233/471] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:01.626486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79255d83-e8bd-4ac3-9ce9-5e817f4d2e22 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:01.642919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 239/472] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:01.643946 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ed9ba-1e21-42c3-ba9f-de57738a3331 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.644229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ed9ba-1e21-42c3-ba9f-de57738a3331 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.644703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ed9ba-1e21-42c3-ba9f-de57738a3331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.665996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e49ed9ba-1e21-42c3-ba9f-de57738a3331 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:01.667225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e49ed9ba-1e21-42c3-ba9f-de57738a3331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:01.678125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:01.779528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 240/473] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:01.792267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79255d83-e8bd-4ac3-9ce9-5e817f4d2e22 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:01.828521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:01.829154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:01.829428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:01.829693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:01.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:01.832478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:01.856292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 234/474] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:46:01 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:01.991683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ed9ba-1e21-42c3-ba9f-de57738a3331 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:02.084551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:02.106438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:02.120614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ed9ba-1e21-42c3-ba9f-de57738a3331 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:02.165822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:02.166473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:02.166753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:02.167025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:02.182021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:02.183814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:02.200693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:02.201653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:02.202484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:02.202484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:02.202484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:02.205348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 241/475] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:02.213801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:02.215618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:02.238306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e4eaf-ed40-456b-83ec-56dbc4983e80 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:02.277047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:02.277751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:02.278126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:02.278495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:02.279304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:02.281211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:02.289359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.sql.core [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 15:46:02.289359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 15:46:02.289359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [parameters: {'id': 'd8305f67bb9e46e78cae0349722ce28c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 15:46:02.289359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81367) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application conn.query(q) Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result.read() Oct 03 15:46:02.308703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return ref.to_dict() Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 15:46:02.309877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application next(self.gen) Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application yield resource Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application next(self.gen) Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise self.value Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application session.commit() Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 15:46:02.310781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.session.flush() Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application compat.raise_( Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 15:46:02.311639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 15:46:02.312535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application conn.query(q) Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result.read() Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 15:46:02.313367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [parameters: {'id': 'd8305f67bb9e46e78cae0349722ce28c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:02.314163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 15:46:02.314963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:02.317548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 235/476] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 15:46:02 2022] POST /identity/v3/roles => generated 144 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:02.318494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:02.355090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e0f84-2965-4d60-be14-0c5987e6bd09 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:02.406427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:02.423732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:02.424355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:02.424635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:02.424928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:02.426196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:02.426499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:02.434710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:02.467212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 242/477] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:02.472019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:02.495138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384d24d-c49a-4733-9100-a5a116d13b68 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:02.598424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:02.599332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:02.599609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:02.599875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:02.602739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a195b3-478d-40ba-9ac2-c6cb9efa7dbc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:02.628626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:02.640091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:02.641056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:02.641851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 236/478] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:02.646491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739d42f-680f-47a6-8796-774c10cd5b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:02.647126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739d42f-680f-47a6-8796-774c10cd5b68 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:02.647405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739d42f-680f-47a6-8796-774c10cd5b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:02.647678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739d42f-680f-47a6-8796-774c10cd5b68 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:02.648331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739d42f-680f-47a6-8796-774c10cd5b68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:02.649871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739d42f-680f-47a6-8796-774c10cd5b68 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:02.661981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:02.681238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:02.695425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 243/479] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:46:02 2022] POST /identity/v3/users => generated 249 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:46:02.809934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56036b44-05f9-45db-93be-fd8a6ddff5f7 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:02.867551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ca45c-9eda-45cf-b610-81451550b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:02.868094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ca45c-9eda-45cf-b610-81451550b2ff None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:02.868377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ca45c-9eda-45cf-b610-81451550b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:02.868650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ca45c-9eda-45cf-b610-81451550b2ff None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:02.869374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ca45c-9eda-45cf-b610-81451550b2ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:02.869678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ca45c-9eda-45cf-b610-81451550b2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:02.871187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ca45c-9eda-45cf-b610-81451550b2ff None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:02.885663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 237/480] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:03.973112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0407c-c111-4a34-8d27-c467263de5d7 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:03.974365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0407c-c111-4a34-8d27-c467263de5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:03.974872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0407c-c111-4a34-8d27-c467263de5d7 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:03.976654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 244/481] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:03.982352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb542-1b75-40de-b021-1b54b9a5ec3a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:03.983157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb542-1b75-40de-b021-1b54b9a5ec3a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:03.983625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eb542-1b75-40de-b021-1b54b9a5ec3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:04.002924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-bc4eb542-1b75-40de-b021-1b54b9a5ec3a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:04.005242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bc4eb542-1b75-40de-b021-1b54b9a5ec3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:04.104287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 238/482] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:04.424949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cec89-8ad5-441e-84c3-f2a3d112018a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:04.425857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cec89-8ad5-441e-84c3-f2a3d112018a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:04.426332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cec89-8ad5-441e-84c3-f2a3d112018a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:04.428148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 245/483] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:04.433132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6691d-c44c-458e-94a9-2ed281556a38 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:04.433421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6691d-c44c-458e-94a9-2ed281556a38 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:04.433693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6691d-c44c-458e-94a9-2ed281556a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:04.445604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:04.462559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1ca6691d-c44c-458e-94a9-2ed281556a38 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:04.463763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1ca6691d-c44c-458e-94a9-2ed281556a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:04.496830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cec89-8ad5-441e-84c3-f2a3d112018a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:04.560964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 239/484] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:04.568014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:04.665790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cec89-8ad5-441e-84c3-f2a3d112018a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:04.714015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:04.714988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:04.715489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:04.715987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:04.724009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b25b-d648-437b-8eba-578d31d79d7b None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:04.724301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b25b-d648-437b-8eba-578d31d79d7b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:04.724578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b25b-d648-437b-8eba-578d31d79d7b None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:04.728784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 240/485] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:04.733324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:04.738724 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04b57-3618-462c-b5a3-5e2cfee32eda None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:04.739145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04b57-3618-462c-b5a3-5e2cfee32eda None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:04.739456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f04b57-3618-462c-b5a3-5e2cfee32eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:04.739841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:04.740696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:04.741503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:04.744286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 246/486] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:04.752931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:04.782471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c3f04b57-3618-462c-b5a3-5e2cfee32eda None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:04.784317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c3f04b57-3618-462c-b5a3-5e2cfee32eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:04.904185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 241/487] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:04.904185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f04b57-3618-462c-b5a3-5e2cfee32eda None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:04.932215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8cbc58-d7e8-443b-aea9-1e6fadcfcfd0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:04.945696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.003817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.005916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.008150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.009196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.011990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.012910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:05.013392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.015011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.034818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 247/488] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:05.040610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.082515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f04b57-3618-462c-b5a3-5e2cfee32eda None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.129565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.131998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.133190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.133876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.157317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:05.170576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:05.171488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:05.172268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:05.175553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 242/489] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:05.181820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.227948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dce5a-c1f3-4dcd-b99a-6656c07edfc7 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.269861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.270685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.270970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.271239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.283291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.284910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:05.300990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:05.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:05.303142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:05.303142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:05.303142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:05.303142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 248/490] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:05.303362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.325333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f557e-60c4-425b-b439-c15c0ba39548 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.328027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.354905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.368350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.369323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.369865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.370329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.371213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:05.371730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.373713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.385094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 243/491] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:05.386248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.421359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48e5e-c677-4cca-abdf-235ee4b5a6e7 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.468663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.469547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.469830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.470105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.491329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:05.497930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:05.498831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:05.499612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:05.500924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 249/492] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:05.501467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.539349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2159510-b198-4e92-8d93-8ffd779d495a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.580918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.582042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.582520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.582994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.583902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:05.584518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.586528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.599081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 244/493] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:05.600216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.615576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.650807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f66d19-317a-4919-8855-4f018745453a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.666137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.701334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab018c24-4a13-48b5-8c28-185958c548a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.702074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab018c24-4a13-48b5-8c28-185958c548a3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.702411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab018c24-4a13-48b5-8c28-185958c548a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.702850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab018c24-4a13-48b5-8c28-185958c548a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.719703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab018c24-4a13-48b5-8c28-185958c548a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.722635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab018c24-4a13-48b5-8c28-185958c548a3 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-ab018c24-4a13-48b5-8c28-185958c548a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:05.741798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:05.742872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:05.743709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:05.743709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:05.743709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:05.743709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 250/494] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:05.743936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d57f4-d68c-4ce7-ac39-351c4153a18f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.744360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434784a5-5a55-4037-9597-ca5b9fe91aee None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.744457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434784a5-5a55-4037-9597-ca5b9fe91aee None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.744742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434784a5-5a55-4037-9597-ca5b9fe91aee None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 251/495] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:05.746689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434784a5-5a55-4037-9597-ca5b9fe91aee None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.779920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a1ae7-1deb-4ea9-af1a-a7a97875ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.780517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1ae7-1deb-4ea9-af1a-a7a97875ba09 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.780826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1ae7-1deb-4ea9-af1a-a7a97875ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.781094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a1ae7-1deb-4ea9-af1a-a7a97875ba09 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.781752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a1ae7-1deb-4ea9-af1a-a7a97875ba09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:05.782047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a1ae7-1deb-4ea9-af1a-a7a97875ba09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.783414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a1ae7-1deb-4ea9-af1a-a7a97875ba09 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.792053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 245/496] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:05.793575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa5d0-b696-4b78-92ac-2bfece5ca31f None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.793863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa5d0-b696-4b78-92ac-2bfece5ca31f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.794135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaa5d0-b696-4b78-92ac-2bfece5ca31f None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.799421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 246/497] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:05.799421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eaa5d0-b696-4b78-92ac-2bfece5ca31f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:05.855205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434784a5-5a55-4037-9597-ca5b9fe91aee None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.886454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.892633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6c358-fd49-4156-9e16-d595259ac0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.893400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6c358-fd49-4156-9e16-d595259ac0ea None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.893711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6c358-fd49-4156-9e16-d595259ac0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.893992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6c358-fd49-4156-9e16-d595259ac0ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.901283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6c358-fd49-4156-9e16-d595259ac0ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.902715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6c358-fd49-4156-9e16-d595259ac0ea None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.903168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eaa5d0-b696-4b78-92ac-2bfece5ca31f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:05.913671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a1e6c358-fd49-4156-9e16-d595259ac0ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:05.916413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:05.917517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:05.918401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 252/498] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:05.918978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5112ee-930b-4046-8526-64a147c513e9 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.919095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5112ee-930b-4046-8526-64a147c513e9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.919372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5112ee-930b-4046-8526-64a147c513e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.937694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0f5112ee-930b-4046-8526-64a147c513e9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:05.938901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0f5112ee-930b-4046-8526-64a147c513e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:05.943213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:05.943819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:05.944093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:05.944358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:05.945039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:05.945338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:05.947108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:05.959253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 247/499] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:05.960388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.005328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 253/500] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:06.006708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74185a-493b-4674-8f01-2aa9a52fbf96 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.006997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74185a-493b-4674-8f01-2aa9a52fbf96 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.007268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74185a-493b-4674-8f01-2aa9a52fbf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.024971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bd74185a-493b-4674-8f01-2aa9a52fbf96 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:06.026158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bd74185a-493b-4674-8f01-2aa9a52fbf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:06.064869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baec958-f0d7-4ed9-897b-b772a0b57b1f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.104821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 254/501] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:06.104821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74185a-493b-4674-8f01-2aa9a52fbf96 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.108250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.108898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.109175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.109443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.121507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.123109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:06.138357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:06.139339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:06.140180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:06.140180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:06.140180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:06.144779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 248/502] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:06.145631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.161463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.180369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.231805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74185a-493b-4674-8f01-2aa9a52fbf96 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.271318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.271987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.272262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.272526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.273283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:06.273827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.276144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.283928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028668d2-40ea-467d-a25e-b1da6b7261df None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.319400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 255/503] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:06.319771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.325939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.326927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.327273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.327622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] PATH_INFO: `/v3/users/glance` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.336620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.338240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:06.352794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:06.353686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:06.354574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 249/504] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:06.355729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.428236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85187998-151d-4f78-9c7c-61060d9033de None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.437436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.440801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.460182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a059b1-7378-45f5-9eb0-22300b65c3be None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.462570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.463176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.463459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.463732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.464376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:06.464677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.466095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.478303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 256/505] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:06.479100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.498031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.498992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.499490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.499992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.511542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:06.516838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:06.517911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 250/506] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:06.518897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.581719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2db0a-d179-40e4-aa3f-b45d446bcdc8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.618026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.618617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adda30f-99e0-427b-8dbc-4af35e2b1206 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.618716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.618891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.619161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.651700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.652297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.652426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.652570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.652871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.653609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:06.653905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.655540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.680710 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.687855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.689354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.692550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 251/507] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:06.693394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.788461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 257/508] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:06.789257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.796916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac7aa0-61b9-4836-8393-4c2354b97533 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.831756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.832547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.832851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.833119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.842871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.844328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.860164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:06.860746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:06.861772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:06.862905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:06.862905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:06.862905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:06.862905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 252/509] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:06.863352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.899047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5c368-f379-4d81-890d-9530a4974fcc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:06.912454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:06.933835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:06.934462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:06.934738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:06.935008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:06.942091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:06.943567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:06.954933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:06.955843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:06.957448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 258/510] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:06.958031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:06.966545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e2158-b4e0-4c22-9c38-9f8ba5d63337 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.000103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.000704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.001005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.001273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.001937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:07.002239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.004770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.012539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 253/511] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:07.013348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.058951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d98191-ee84-4beb-af89-2e8290f77685 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.084997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.093441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.094044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.094315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.094581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.117192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149af90-dad6-4a5d-a14d-1fd6cbc34204 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.131430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.152893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.153516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.153792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.154058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.164032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.165078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.165542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.166597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:07.181329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:07.182382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:07.183210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:07.183210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:07.183210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:07.183210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 254/512] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:07.183210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.267494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 259/513] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:07.268505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.280135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b276-2b39-444a-8f43-af45a53b9f5e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.299212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.316403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.317047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.317321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.317597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.318232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:07.318530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.319910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.332215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 255/514] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:07.333389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.359506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.374288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f21ae6-93bd-4518-8979-9510c7deed81 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.407693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.408289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.408561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.408855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.409613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:07.409964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.411584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.432262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 260/515] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:07.432262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.436384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cdce2-6352-4b64-83cf-3f98bb10a759 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.469757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.470371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.470643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.470911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.481803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.483290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:07.498693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:07.499600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:07.500435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:07.500435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:07.500435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:07.500435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 256/516] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:07.500641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.512900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.528443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fcb80-c640-4f66-b818-dea9a84adbd6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.561557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.562160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.562439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.562711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.563349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:07.563652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.565153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.577110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 261/517] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:07.578038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.578481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.606771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db7be-a1c3-4ca2-9776-42b77e75047c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.640631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.641278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.641552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.641818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.652241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.653739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:07.667501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:07.668428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:07.669759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:07.669759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:07.669759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:07.669759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 257/518] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:07.669759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.681331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8796a-c46d-43aa-8226-fb8fdd2e1568 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.714610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.715205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.715479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.715751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.716387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.717861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.741858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 262/519] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:46:07 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:07.742659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.752919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.785232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f1f56-5a06-4916-ac75-bb7b5ac87ba8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.814545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:07.824480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.825237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.825518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.825868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.867357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb53e8-16af-443f-91bc-9b659d05b69a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:07.904477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.906124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.906760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:07.907376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:07.907647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:07.907909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:07.908550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:07.908877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:07.911075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:07.923965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 263/520] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:07.925377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:07.996998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.031692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 258/521] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:08.032544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.044544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529b55-7777-488a-9cb7-75ee2fe0d40b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.064005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.081445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.082194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.082274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.082495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.089315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.090754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:08.103623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:08.104499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 264/522] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:08.110219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.150605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6f5a1-fa44-4dce-84b3-267f707a5396 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.186090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.186694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.186976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.187225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.187875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:08.188178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.189600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.203015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 259/523] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:08.205901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.225449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5288af-d288-45ed-a1d9-932dc3909e32 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.248543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.263695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.264376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.264596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.264897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.273473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.275396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:08.287917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:08.289104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 265/524] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:08.290385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.311210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.327840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c61b58-17bb-42c7-b803-fd1db642d37b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.371945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.372759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.372868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.373119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.384336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.385830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.398842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031348a0-7aed-4310-9549-0db24b89e987 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:08.400984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:08.401876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:08.402746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:08.402746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:08.402746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:08.402746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 260/525] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:08.402971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.439599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.440221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.440525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.440823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.441523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:08.441830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.443286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.461542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 266/526] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:08.467086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.510054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc328c-3f25-4455-b769-1c34154b6e29 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.514803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.546649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.547280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.547556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.547827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.555762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.557275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.559847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:08.570383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:08.571275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:08.572112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 261/527] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:08.572571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.583377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de362f4-506f-4ab4-a063-eaa79b4a3244 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.620408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.621048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.621330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.621599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.622330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:08.622633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.624070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.642266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 267/528] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:08.642266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.686782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bd1ab-a490-4f12-b3af-3e1a8d2d3cd7 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.722855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.723586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.724213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.724581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.725372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:08.725777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.727291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.739787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 262/529] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:08.741240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.750740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.754761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d2770-9df1-43ce-9fdd-e8b4b5d0f28d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.791816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.792624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.793009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.793344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.794140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=ed959e4bdaf44132b05ccaec68c41172) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:08.794510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.796008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.801625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 268/530] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/role_assignments?user.id=ed959e4bdaf44132b05ccaec68c41172&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:08.802370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:08.819693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:08.891660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f731d-fa06-44be-93db-cacbc76f0ddc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.927635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109d277-7422-4511-aae8-d99eef0536f6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:08.944871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.946018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.946467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.946901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.948413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:08.948920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:08.950568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:08.971362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:08.972853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:08.973286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:08.973953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:08.979909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 263/531] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:08.979909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:09.018219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:09.065763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.067248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.087466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3662b5c-e45f-4cb5-bd49-f576d5d1f896 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:09.111306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:09.129794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:09.130705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:09.131213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:09.131766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:09.139284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.141068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:09.156703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:09.158125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:09.159343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:09.166351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 264/532] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:09.167559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:09.188805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 269/533] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:09.190252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:09.262418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:09.296425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471cad9-aa2c-401c-9c09-65038aa5ffce None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:09.311756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfddc1-deba-4f45-ab9f-81df03a88445 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:09.347822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:09.348438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:09.349002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:09.349255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:09.350097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=022516f130ec4ad6a9b7c46b417a5690) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:09.350403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.352314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.353942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:09.354819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:09.355296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:09.355771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:09.356713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:09.357259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.364762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 265/534] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:09 2022] GET /identity/v3/role_assignments?user.id=022516f130ec4ad6a9b7c46b417a5690&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:09.364762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:09.366391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.387510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 270/535] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:09.393879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:09.409022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:09.473356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899cb53-4ea8-4ade-b3c2-4620c9a9a9de None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:09.513269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:09.514239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:09.514754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:09.515229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:09.528330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.530227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:09.547195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:09.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:09.549001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:09.549001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:09.549001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:09.551502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 266/536] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:09.557280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:09.625628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3dfc5-50ff-4014-9d0d-0413cbf1ecf6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:09.645954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:09.661529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:09.662140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:09.662420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:09.662693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:09.663475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=4396302927da47a3a80eae4c50fdb266) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:09.663725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.665468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.671310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 271/537] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:09 2022] GET /identity/v3/role_assignments?user.id=4396302927da47a3a80eae4c50fdb266&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:09.693892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:09.701690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc42f0-fa77-42be-815a-45868b21af5c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:09.742941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:09.743988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:09.744523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:09.745041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:09.745918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:09.746442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.748160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.765326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 267/538] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:09.773880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:09.917019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66589e2b-0810-4d12-abd5-50eb7b2f5319 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:09.956972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354bfbb-10b5-40f3-911e-9e13451696ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:09.957757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354bfbb-10b5-40f3-911e-9e13451696ef None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:09.958037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354bfbb-10b5-40f3-911e-9e13451696ef None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:09.958311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354bfbb-10b5-40f3-911e-9e13451696ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:09.965467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354bfbb-10b5-40f3-911e-9e13451696ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:09.966924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354bfbb-10b5-40f3-911e-9e13451696ef None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f354bfbb-10b5-40f3-911e-9e13451696ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:09.979188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:09.980046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:09.980892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 272/539] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:09.987626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:10.046762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc17166-2e9c-43ca-9fbd-1c0286ead9f9 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:10.046946 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc17166-2e9c-43ca-9fbd-1c0286ead9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:10.047252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc17166-2e9c-43ca-9fbd-1c0286ead9f9 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:10.048969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 273/540] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:10.055088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc97505-c998-418e-abdf-1164299b898b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:10.055373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc97505-c998-418e-abdf-1164299b898b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:10.055648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc97505-c998-418e-abdf-1164299b898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:10.074771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ecc97505-c998-418e-abdf-1164299b898b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:10.075974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ecc97505-c998-418e-abdf-1164299b898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:10.128221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edfbfa-ef18-4366-a40d-4ead21a5dbdb None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:10.146387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:10.174908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:10.176133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:10.176414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:10.176685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:10.177569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:10.177882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:10.179467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:10.187163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 274/541] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:46:10.199624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 268/542] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:10.205368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc97505-c998-418e-abdf-1164299b898b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:10.338789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc97505-c998-418e-abdf-1164299b898b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:10.380373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:10.381019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:10.381300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:10.381571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:10.382313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=2d007a3d205345aebcc36a36d039da2d) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:10.382617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:10.384071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:10.390230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 275/543] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:10 2022] GET /identity/v3/role_assignments?user.id=2d007a3d205345aebcc36a36d039da2d&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:10.528641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:10.548820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:10.673805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16684b62-c5c9-4040-9872-b12463c2076d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:10.711775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:10.712811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:10.713451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:10.713950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:10.726246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:10.731323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:10.732175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:10.732976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:10.735599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 269/544] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:10.742612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:10.803902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:10.865825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e33f4-2666-4651-a263-6edf97b99a2f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:10.903938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:10.904619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:10.904927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:10.905199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:10.905867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:10.906166 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:10.907551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:10.922100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 276/545] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:10.926038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:11.026881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:11.058474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ca9e8-0580-4391-89ae-a1aa06a02918 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:11.106511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:11.107147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:11.107423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:11.107692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:11.123510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:11.125027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:11.140178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:11.141193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:11.142042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:11.142042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:11.142042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:11.145960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 270/546] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:11.147389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:11.252616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ff0ce-c564-4455-aa1f-8d27cfc9d72f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:11.255133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:11.289012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:11.289618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:11.289895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:11.290163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:11.290803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:11.291102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:11.292490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:11.308854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 277/547] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:11.311229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:11.432281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fab895-3416-4001-b107-ad8d2ca36577 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:11.463441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:11.473586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bccba-8bd1-421e-9f3a-22dfbe190d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:11.475170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bccba-8bd1-421e-9f3a-22dfbe190d79 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:11.475666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bccba-8bd1-421e-9f3a-22dfbe190d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:11.476482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bccba-8bd1-421e-9f3a-22dfbe190d79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:11.486052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bccba-8bd1-421e-9f3a-22dfbe190d79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:11.487722 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bccba-8bd1-421e-9f3a-22dfbe190d79 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-011bccba-8bd1-421e-9f3a-22dfbe190d79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:11.499534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:11.500976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:11.502058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:11.504355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 271/548] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:11.507096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:11.527598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806aa3e3-2dfb-4861-839e-77afdc22e79d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:11.527884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806aa3e3-2dfb-4861-839e-77afdc22e79d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:11.528138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806aa3e3-2dfb-4861-839e-77afdc22e79d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:11.529945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 272/549] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:11.540703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31bf71-3be1-454a-8da3-706d8b3ecfbe None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:11.541002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31bf71-3be1-454a-8da3-706d8b3ecfbe None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:11.541273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31bf71-3be1-454a-8da3-706d8b3ecfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:11.576523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-0f31bf71-3be1-454a-8da3-706d8b3ecfbe None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:11.579687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0f31bf71-3be1-454a-8da3-706d8b3ecfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:11.690192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 273/550] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:11.699157 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5e155-fabc-4e63-aa5e-f6acbe445a6c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:11.759923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:11.760464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:11.760755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:11.761031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:11.761774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:11.762079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:11.763514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:11.788352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 278/551] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 15:46:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:11.793996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31bf71-3be1-454a-8da3-706d8b3ecfbe None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:11.951365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31bf71-3be1-454a-8da3-706d8b3ecfbe None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:11.974325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:12.000134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:12.001150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.001636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.002137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.080191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:12.082241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:12.204944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 274/552] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:12.213914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:12.220482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:12.246638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:12.274460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:12.362030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d2d87-1bb8-4602-ab11-826bce5c4884 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:12.388518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1a480-35e1-4ff4-b4d3-1b44545d954e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:12.408233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4544aac-c7c0-40b3-9847-9d2d754d22d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:12.408921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544aac-c7c0-40b3-9847-9d2d754d22d4 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.409901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544aac-c7c0-40b3-9847-9d2d754d22d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.410184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544aac-c7c0-40b3-9847-9d2d754d22d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.425696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4544aac-c7c0-40b3-9847-9d2d754d22d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-d4544aac-c7c0-40b3-9847-9d2d754d22d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:12.432469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:12.434594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:12.437625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:12.439129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7272fd5-b792-4088-b262-c4e3e0bf2ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:12.439129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7272fd5-b792-4088-b262-c4e3e0bf2ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.439129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7272fd5-b792-4088-b262-c4e3e0bf2ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.439129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7272fd5-b792-4088-b262-c4e3e0bf2ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.439432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 275/553] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:12.439432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1f012-c3df-4aab-a558-9e958a6a0b96 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.439432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1f012-c3df-4aab-a558-9e958a6a0b96 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.439432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1f012-c3df-4aab-a558-9e958a6a0b96 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.448759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 276/554] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:12.448759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1f012-c3df-4aab-a558-9e958a6a0b96 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:12.460196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7272fd5-b792-4088-b262-c4e3e0bf2ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:12.462315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7272fd5-b792-4088-b262-c4e3e0bf2ac2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f7272fd5-b792-4088-b262-c4e3e0bf2ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:12.482009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:12.483926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:12.485587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:12.485587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:12.485587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:12.485587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 279/555] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:12.485587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e11c0-7d2d-48e3-8cd4-7bcf8a3ab1ab None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.485587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e11c0-7d2d-48e3-8cd4-7bcf8a3ab1ab None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.485587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e11c0-7d2d-48e3-8cd4-7bcf8a3ab1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.512080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ef3e11c0-7d2d-48e3-8cd4-7bcf8a3ab1ab None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:12.513396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ef3e11c0-7d2d-48e3-8cd4-7bcf8a3ab1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:12.546712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:12.573051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1f012-c3df-4aab-a558-9e958a6a0b96 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:12.590402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 280/556] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:12.591346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e11c0-7d2d-48e3-8cd4-7bcf8a3ab1ab None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:12.599228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:12.618292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0c038-b88a-4556-a8d2-4d361d8d7489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:12.619174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c038-b88a-4556-a8d2-4d361d8d7489 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.619715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c038-b88a-4556-a8d2-4d361d8d7489 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.620136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c038-b88a-4556-a8d2-4d361d8d7489 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.621041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0c038-b88a-4556-a8d2-4d361d8d7489 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:12.621490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0c038-b88a-4556-a8d2-4d361d8d7489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:12.623488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0c038-b88a-4556-a8d2-4d361d8d7489 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:12.634004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 277/557] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:12.635442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96286b-31bb-4951-800f-f553c098069b None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.635855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96286b-31bb-4951-800f-f553c098069b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.636254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96286b-31bb-4951-800f-f553c098069b None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.641889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 278/558] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:12.643025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96286b-31bb-4951-800f-f553c098069b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:12.713653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e11c0-7d2d-48e3-8cd4-7bcf8a3ab1ab None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:12.755039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb2969-9f1c-422d-a50d-22fe57136534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:12.755656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2969-9f1c-422d-a50d-22fe57136534 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.755931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2969-9f1c-422d-a50d-22fe57136534 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.756191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb2969-9f1c-422d-a50d-22fe57136534 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.757095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb2969-9f1c-422d-a50d-22fe57136534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:12.757837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb2969-9f1c-422d-a50d-22fe57136534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:12.759678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb2969-9f1c-422d-a50d-22fe57136534 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:12.776878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 281/559] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:12.778159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67bb6-db03-4796-8f68-6183d5ae83ff None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.778439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67bb6-db03-4796-8f68-6183d5ae83ff None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.778707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67bb6-db03-4796-8f68-6183d5ae83ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.789217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96286b-31bb-4951-800f-f553c098069b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:12.803461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-8ed67bb6-db03-4796-8f68-6183d5ae83ff None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:12.804747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8ed67bb6-db03-4796-8f68-6183d5ae83ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:12.829172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:12.829803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:12.830186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:12.830460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:12.840324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:12.843359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:12.845860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:12.862853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:12.863816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:12.864712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:12.864712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:12.864712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:12.868828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 279/560] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:12.869629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:12.876936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:12.887703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 282/561] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:12.888564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed67bb6-db03-4796-8f68-6183d5ae83ff None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.007787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed67bb6-db03-4796-8f68-6183d5ae83ff None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.014161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325fee5-5207-4d7b-960f-e1ee37b34888 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.046340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.046954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.047233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.047505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.048155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:13.048458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.050990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.064175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.064839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.065141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.065440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.068136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 283/562] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:13.069713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.074725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.075672 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:13.088543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:13.091523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 280/563] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:13.093786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.132908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.151431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.179327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1dc42-8161-42f4-b76e-0690b349886f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.204399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802b97b-1f45-4b47-ab72-d28f210de556 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.211742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64add1ec-6f1b-42f3-bf16-7d353d0e1f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.212363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add1ec-6f1b-42f3-bf16-7d353d0e1f84 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.212637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add1ec-6f1b-42f3-bf16-7d353d0e1f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.212933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64add1ec-6f1b-42f3-bf16-7d353d0e1f84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.224559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64add1ec-6f1b-42f3-bf16-7d353d0e1f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-64add1ec-6f1b-42f3-bf16-7d353d0e1f84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:13.229687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:13.230643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:13.231490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 284/564] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:13.232413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12e507-d7a5-4a51-b9d2-21d68bc2182c None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.232413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12e507-d7a5-4a51-b9d2-21d68bc2182c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.232413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12e507-d7a5-4a51-b9d2-21d68bc2182c None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.233637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 285/565] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:13.234325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12e507-d7a5-4a51-b9d2-21d68bc2182c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.246554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.247188 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.247463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.247737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.255246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:13.267880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:13.268973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:13.269948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 281/566] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:13.270537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.346445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12e507-d7a5-4a51-b9d2-21d68bc2182c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.360655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.371315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3774b-48ea-4be6-ad48-f453ff5afa65 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.381420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.382021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.382291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.382561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.383293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:13.383538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.384206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.385150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.397315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 286/567] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:13.398182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.406743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54eb423-8197-43b5-b7ed-9bbe775798f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.407367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eb423-8197-43b5-b7ed-9bbe775798f9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.407643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eb423-8197-43b5-b7ed-9bbe775798f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.407916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eb423-8197-43b5-b7ed-9bbe775798f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.418478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54eb423-8197-43b5-b7ed-9bbe775798f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-e54eb423-8197-43b5-b7ed-9bbe775798f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:13.423666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:13.424565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 282/568] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:13.426877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dfcff-511a-4698-99f0-fadaa4bf434f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.427779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dfcff-511a-4698-99f0-fadaa4bf434f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.427779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dfcff-511a-4698-99f0-fadaa4bf434f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.445950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6f7dfcff-511a-4698-99f0-fadaa4bf434f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:13.447136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6f7dfcff-511a-4698-99f0-fadaa4bf434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:13.513162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 283/569] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:13.513992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7dfcff-511a-4698-99f0-fadaa4bf434f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.517069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ec567-6ef3-4390-bb80-4e52f1d5ae9b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.555965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.556705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.557129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.557516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.558431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:13.558840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.561062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.575122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 287/570] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:13.575122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.609423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.634973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7dfcff-511a-4698-99f0-fadaa4bf434f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.636428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.670613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.671329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.671609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.671882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.672618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:13.672955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.674401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.684515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224aec0-19e3-47ce-8dd7-8319c277d338 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.692673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 284/571] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:13.693353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.724287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.725532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.725978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.726386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.727363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=7701e2653e4b4ca7a842887c0526862f) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:13.727915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.729593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.735811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 288/572] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/role_assignments?user.id=7701e2653e4b4ca7a842887c0526862f&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:13.736905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.823341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1e63a-ddd9-458a-af01-f8459a768872 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.868102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.871466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65234038-3ea5-45d2-9bd2-3ffedc29b633 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:13.871749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.872369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.872648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.872967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.875100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:13.875100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.876079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.891259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 285/573] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:13.891259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:13.915430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:13.916072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:13.916349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:13.916622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:13.918846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:13.930919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:13.932775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:13.947701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:13.948752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:13.949690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:13.949690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:13.949690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:13.951784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 289/574] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:13.952936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.008655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2a52-1c95-4b77-96d8-dd2db7011aa4 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.044421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.044931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.045334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.045608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.056606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.058126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.061162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b4b84-2bb6-48f2-b3ba-ee9aa34f698f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:14.073881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:14.074842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:14.075706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:14.075706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:14.075706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:14.075706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 286/575] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:14.075942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.103668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.104383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.104737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.105094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.116373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:14.121546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:14.122510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:14.123339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 290/576] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:14.124053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.140108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.163127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.189617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06dea09-f6ae-429c-aee4-890fc9343cfa None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.226572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.227206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.227484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.227751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.233490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a596b5-cb3f-44cd-a5d7-b7681ee48cb0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.238549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.240026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:14.255573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:14.256691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:14.257611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:14.257611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:14.257611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:14.257611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 287/577] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:14.257848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.274575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.275187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.275463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.275730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.276372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:14.276706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.278085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.290222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 291/578] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:14.291030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.368761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.375762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11c6e9-3b19-4f66-ba4c-3f7d802d654d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.401854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d54f5-a641-41e8-8fc6-26ef402feb20 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.403084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.413961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.414585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.414949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.415287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.416012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:14.416484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.418088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.435982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 288/579] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:14.435982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.442957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.443775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.443904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.444110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.444802 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:14.445109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.446549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.459810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 292/580] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:14.459810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.546409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df81a3-9bfc-4be7-bbe5-530ea1e714d6 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.567835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320330ea-72ea-4e47-8db2-5815781837bc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.581376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.582137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.582439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.582707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.583352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:14.583654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.585059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.596057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 289/581] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:14.596989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.611764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.613354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.613833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.614303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.618008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.626033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.628663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:14.642423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:14.643649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 15:46:14.644637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:14.650237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 293/582] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:14.650237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.651243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.707693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9951ca6-0803-4733-83d4-f25ccfb644bf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.742364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.742981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.743255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.743521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.750254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.751716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.761161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcdbaa-7e7f-461a-a60c-53def1b208cc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:14.764141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:14.765084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:14.765991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 290/583] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:14.770925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.796714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.797366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.797643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.797920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.808526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.810044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:14.824492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:14.825603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:14.826492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:14.826492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:14.826492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:14.830091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 294/584] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:14.830926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.879366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695cf16-3dce-4bf5-8357-747845c0a503 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.880432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.886507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:14.916852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.917485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.917761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.918030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.925755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.927239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:14.939650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:14.941304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:46:14.942237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:14.942777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d146c-e32b-4d92-a2b7-07e5f11555c8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:14.946195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 291/585] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:14.947061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:14.977849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:14.978381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:14.978766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:14.979039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:14.979891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:14.980198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:14.981658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.007226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 295/586] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:15.007226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.056491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dccd2-466a-4121-bfd2-0510f29ecffe None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.091314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.091946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.092226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.092496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.093226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:15.093538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.094978 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.108030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 292/587] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:15.113958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.118132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c37b74-2c82-43c5-9ba4-cfbf51ad9442 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.121236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.144904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.159803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.161021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.161302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.161576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.162585 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:15.162907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.164891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.181835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 296/588] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:15.181835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.231454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b24cea-179b-4e21-9476-2a83b39909c1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.265314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.265910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.266180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.266441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.267172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:15.267470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.269462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.287329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2897-57ed-4032-9d80-d166015393ae None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.288118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 293/589] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:15.288941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.327425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.328053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.328326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.328646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.341113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.342679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:15.358990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:15.360163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:15.361560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:15.361560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:15.361560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:15.361560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 297/590] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:15.362061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.385606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.392187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.411705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30cf22-6814-48ed-81fd-ba41959fb4a5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.449341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.450000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.450298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.450665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/ed959e4bdaf44132b05ccaec68c41172/roles/e3ae1cd82e064f38a35cc76b525ffbbf` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.466514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=ed959e4bdaf44132b05ccaec68c41172, role_id=e3ae1cd82e064f38a35cc76b525ffbbf)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.468230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.474935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7629f-a99b-46db-85f2-5577bdb26956 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.496837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 294/591] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:15 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/ed959e4bdaf44132b05ccaec68c41172/roles/e3ae1cd82e064f38a35cc76b525ffbbf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:15.496837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.533741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.535086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.535368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.535641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.547124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.548612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:15.567269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:15.568261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 15:46:15.569220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:15.570312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 298/592] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:15.570907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.621689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee8661-504e-4f6a-aa5f-f5f68ed3c4e5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.668620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.669447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.669731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.670073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.673170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.687264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.688522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.690765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:15.708997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:15.709991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:15.711296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:15.711296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:15.711296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:15.711296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 295/593] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:15.711296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.719349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffe544-f5cb-4cb6-808e-c62f3a48f686 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.756611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.757178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.757457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.757731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.758390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:15.758692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.760199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.774628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 299/594] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:15.775500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.833793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3b3cc-5c9f-4f15-b95e-aeda07635469 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.874123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.875659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.875960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.876236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.877068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:15.877383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.878985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.902322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 296/595] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:15.903642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:15.917868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c20e38-676e-4335-83aa-3b11a68c2f07 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:15.952259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.957043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f27e144-8765-4a9f-995d-19b9e90916aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:15.957685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e144-8765-4a9f-995d-19b9e90916aa None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.957986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e144-8765-4a9f-995d-19b9e90916aa None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.958213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f27e144-8765-4a9f-995d-19b9e90916aa None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.958856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27e144-8765-4a9f-995d-19b9e90916aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:15.959155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27e144-8765-4a9f-995d-19b9e90916aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:15.960525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f27e144-8765-4a9f-995d-19b9e90916aa None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:15.963062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:15.974968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 300/596] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:15.976196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d193f-9c65-41ea-bc09-febfb0b29bfa None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:15.976478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d193f-9c65-41ea-bc09-febfb0b29bfa None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:15.976766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d193f-9c65-41ea-bc09-febfb0b29bfa None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:15.981044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 301/597] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:15.981988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d193f-9c65-41ea-bc09-febfb0b29bfa None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.012473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a84a-5864-4d7a-86e0-9310bd784ee6 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.052398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.053056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.053331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.053599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.060287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.061782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:16.072883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:16.073771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:16.074652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 297/598] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:16.075133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.099439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d193f-9c65-41ea-bc09-febfb0b29bfa None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.137828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47ac25-b829-4cbe-83c7-4716a0ce06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.138719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47ac25-b829-4cbe-83c7-4716a0ce06f0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.139136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47ac25-b829-4cbe-83c7-4716a0ce06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.139544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47ac25-b829-4cbe-83c7-4716a0ce06f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.151818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47ac25-b829-4cbe-83c7-4716a0ce06f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.153511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47ac25-b829-4cbe-83c7-4716a0ce06f0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2b47ac25-b829-4cbe-83c7-4716a0ce06f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:16.170541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:16.171709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:16.172797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:16.172797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:16.172797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:16.179918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 302/599] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:16.179918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf3f8f-03b7-4bb9-8839-872fdcadfd59 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.179918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf3f8f-03b7-4bb9-8839-872fdcadfd59 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.179918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf3f8f-03b7-4bb9-8839-872fdcadfd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.188666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665fa53-9674-45bd-9bdb-21e1fb751efb None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.191643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:16.200861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-78bf3f8f-03b7-4bb9-8839-872fdcadfd59 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:16.202038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-78bf3f8f-03b7-4bb9-8839-872fdcadfd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:16.227551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.228263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.228600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.228972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.235756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.237890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:16.249617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:16.250526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:16.251398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:16.251819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:16.251890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 298/600] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:16.252802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.283599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 303/601] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:16.285032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf3f8f-03b7-4bb9-8839-872fdcadfd59 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.389299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7d502-097e-415d-8466-20e019fd68e9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.397278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf3f8f-03b7-4bb9-8839-872fdcadfd59 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.432361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.433114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.433460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.433805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.434516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:16.434884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.436386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.436666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.438075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.438525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.439489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.440950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:16.441451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.443047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.450084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 304/602] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:16.451098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.466389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 299/603] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:16.466389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.495623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:16.512614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:16.574881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7abc8c-1289-436b-a7aa-6ada34ef89e9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.590928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b1d85-d99a-43a0-ada3-7ffc392354a5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.619316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.620165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.620454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.621130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.626269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.626861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.627236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.627511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.628211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.628373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:16.628661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.629711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.630101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:16.640768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:16.647437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:16.649286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 300/605] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:16.649814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 305/605] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:16.649887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.649964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.737894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:16.754050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:16.754881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a09c94-72bd-4c13-b38d-3957f53f9b9f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.759256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2e6b5-786a-40f8-ade2-cb4b8a00001e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.793358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.794022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.794323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.794596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/022516f130ec4ad6a9b7c46b417a5690/roles/e3ae1cd82e064f38a35cc76b525ffbbf` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.797350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.798865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.799140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.799407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.810095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=022516f130ec4ad6a9b7c46b417a5690, role_id=e3ae1cd82e064f38a35cc76b525ffbbf)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.811552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.811678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:16.818642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:16.819576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 306/606] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:16.820417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.831626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 301/607] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:16 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/022516f130ec4ad6a9b7c46b417a5690/roles/e3ae1cd82e064f38a35cc76b525ffbbf => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:16.832324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:16.952772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc58cac-cd5a-4b18-afa0-b672ab408382 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.975334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77370d13-ebf2-459e-94d6-b2a31ca4889b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:16.992889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:16.993514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:16.993847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:16.994124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:16.994841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:16.995192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:16.996667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:16.998824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:17.017054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 302/608] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:17.017959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:17.028496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:17.029326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.029609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.029882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/4396302927da47a3a80eae4c50fdb266/roles/e3ae1cd82e064f38a35cc76b525ffbbf` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.046021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=4396302927da47a3a80eae4c50fdb266, role_id=e3ae1cd82e064f38a35cc76b525ffbbf)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:17.047729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:17.063366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 307/609] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:16 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/4396302927da47a3a80eae4c50fdb266/roles/e3ae1cd82e064f38a35cc76b525ffbbf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:17.064450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:17.079859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:17.181259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999fdcf-69e4-4dd7-862c-07ce12aa7e14 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:17.209176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717cf8f-f9dc-4f10-b1f3-476a5a498c3a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:17.262882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:17.263713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.264088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.264448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.265511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:17.267202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:17.269792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:17.281960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:17.282790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.283177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.283541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/2d007a3d205345aebcc36a36d039da2d/roles/e3ae1cd82e064f38a35cc76b525ffbbf` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.287106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 303/610] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:17.293897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:17.312249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=2d007a3d205345aebcc36a36d039da2d, role_id=e3ae1cd82e064f38a35cc76b525ffbbf)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:17.316461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:17.334182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 308/611] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:17 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/2d007a3d205345aebcc36a36d039da2d/roles/e3ae1cd82e064f38a35cc76b525ffbbf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:17.387385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:17.453280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6940b33-81fd-4c23-841e-92b46b134ac9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:17.494520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:17.495463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.495902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.496314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.508439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:17.510232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:17.526587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:17.527778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:17.528915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:17.528915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:17.528915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:17.536312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 304/612] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:17.537972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:17.610552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:17.658213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233c73e-a8d6-4e31-8c1a-063521bbe12f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:17.695200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55c8f4-a0ff-4bfd-9131-e5cb63360715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:17.695821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55c8f4-a0ff-4bfd-9131-e5cb63360715 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.696102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55c8f4-a0ff-4bfd-9131-e5cb63360715 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.696374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55c8f4-a0ff-4bfd-9131-e5cb63360715 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.697056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55c8f4-a0ff-4bfd-9131-e5cb63360715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:17.697358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55c8f4-a0ff-4bfd-9131-e5cb63360715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:17.698736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55c8f4-a0ff-4bfd-9131-e5cb63360715 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:17.711247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 309/613] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:17.717006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:17.864965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec08588-56f4-482c-8d5f-cbda87eaba39 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:17.874490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:17.876289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038dbb2-131f-44c3-97e5-fa0a2918a9c0 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.876850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038dbb2-131f-44c3-97e5-fa0a2918a9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.877331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038dbb2-131f-44c3-97e5-fa0a2918a9c0 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.884040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 310/614] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:17.886052 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c72c1-c9e2-4af4-a55c-f2454a3f8fd0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.886547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c72c1-c9e2-4af4-a55c-f2454a3f8fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.887023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c72c1-c9e2-4af4-a55c-f2454a3f8fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.902859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:17.903491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:17.903764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:17.904030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:17.907675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f04c72c1-c9e2-4af4-a55c-f2454a3f8fd0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:17.909717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f04c72c1-c9e2-4af4-a55c-f2454a3f8fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:17.917599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:17.919109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:17.945176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:17.946198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:17.952024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 305/615] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:17.954919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:18.025445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 311/616] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:18.122407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fef7b7-d6fa-49ce-bc41-119611b6c168 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:18.169929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:18.170535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:18.170808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:18.171070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:18.171807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:18.172103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:18.173576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:18.202509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 306/617] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 15:46:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:18.208380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c72c1-c9e2-4af4-a55c-f2454a3f8fd0 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:18.269055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:18.353449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c72c1-c9e2-4af4-a55c-f2454a3f8fd0 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:18.398770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:18.399791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:18.400274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:18.400768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:18.418184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:18.419941 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:18.437639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:18.438606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:18.439506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:18.439506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:18.439506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:18.441874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 312/618] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:18.448441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:18.448600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:18.490896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:18.568033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804608-7cf5-4990-80f1-5ea8b49a23d1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:18.605029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:18.605981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:18.606551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:18.607010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:18.607857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:18.608363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:18.609980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:18.622159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 307/619] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:18.627091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:18.635171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:18.641047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85f77-0de3-4bd0-8695-3aaa26700ead None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:18.707324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:18.707956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:18.708220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:18.708481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:18.730105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:18.735303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:18.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:18.737263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:18.745309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 313/620] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:18.746174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:18.805113 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:18.810191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ad65-0f3b-4c7a-9586-66a28bbf5f05 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:18.877158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:18.877821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:18.878096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:18.878370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:18.892557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:18.895795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:18.911830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:18.912789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:18.913697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:18.915267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 308/621] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:18.920033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:18.939691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c6f7b-f25e-45c9-ba70-b53931d6a007 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:19.008783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:19.009400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.009674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.009943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.010608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:19.010905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:19.012280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:19.029888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 314/622] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:19.034487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:19.081663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:19.117561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf78f85-c006-41e7-9e77-87c8ee99b3e0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:19.184789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:19.185310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.185584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.185848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.186540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:19.186838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:19.188270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:19.214836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 309/623] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:19.218242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:19.229080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:19.233990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4d715-69c0-4b00-a8ce-57956d368fe6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:19.322739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:19.323659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.324166 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.324643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.341101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f585c40-0421-458a-bc7d-c7cbf89b20b8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:19.351218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:19.353269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:19.385712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a48d6-6126-4f35-8a37-f4d07f3ce193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:19.386373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a48d6-6126-4f35-8a37-f4d07f3ce193 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.386643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a48d6-6126-4f35-8a37-f4d07f3ce193 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.386907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a48d6-6126-4f35-8a37-f4d07f3ce193 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/7701e2653e4b4ca7a842887c0526862f/roles/e3ae1cd82e064f38a35cc76b525ffbbf` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:19.389453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:19.390378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:19.391263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:19.391263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:19.391263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:19.398820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 315/624] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:19.402810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a48d6-6126-4f35-8a37-f4d07f3ce193 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=7701e2653e4b4ca7a842887c0526862f, role_id=e3ae1cd82e064f38a35cc76b525ffbbf)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:19.404014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:19.404569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a48d6-6126-4f35-8a37-f4d07f3ce193 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:19.428552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 310/625] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:19 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/7701e2653e4b4ca7a842887c0526862f/roles/e3ae1cd82e064f38a35cc76b525ffbbf => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:19.507330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:19.527430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e211a3-184f-4701-9d4b-3672e9538939 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.527985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e211a3-184f-4701-9d4b-3672e9538939 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.527985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e211a3-184f-4701-9d4b-3672e9538939 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.529648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 311/626] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:19.535309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59d01-e1ee-43ec-9116-334e62a77a2f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.535999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59d01-e1ee-43ec-9116-334e62a77a2f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.536501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59d01-e1ee-43ec-9116-334e62a77a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.541442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130160e7-49c7-4c5b-8e4e-661451066179 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:19.570742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-97c59d01-e1ee-43ec-9116-334e62a77a2f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:19.572413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-97c59d01-e1ee-43ec-9116-334e62a77a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:19.587642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:19.588313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.588591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.588891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.589603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:19.589902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:19.591349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:19.607780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 316/627] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:19.614307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:19.732709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:19.754222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d57ed-25f5-453d-9132-701f16897334 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:19.758004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 312/628] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:19.759616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a73ab-3150-41e4-81bd-01e44e494d11 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.760118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a73ab-3150-41e4-81bd-01e44e494d11 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.760597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a73ab-3150-41e4-81bd-01e44e494d11 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.762465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 313/629] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:19.768756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28fce-59fe-4016-8d49-6646c404dc47 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.768756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28fce-59fe-4016-8d49-6646c404dc47 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.768756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28fce-59fe-4016-8d49-6646c404dc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.788529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2ae28fce-59fe-4016-8d49-6646c404dc47 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:19.789759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2ae28fce-59fe-4016-8d49-6646c404dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:19.796146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:19.796855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:19.797131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:19.797564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:19.800701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:19.811224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:19.812682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:19.829374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:19.830338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:19.832750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 317/630] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:46:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:19.843995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:19.886828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 314/631] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:20.003183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef364100-aba8-46f4-9fc1-7576fb19fe40 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.054907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c349899-aaa8-4331-a124-a7599d5351de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.055448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c349899-aaa8-4331-a124-a7599d5351de None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.055727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c349899-aaa8-4331-a124-a7599d5351de None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.055998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c349899-aaa8-4331-a124-a7599d5351de None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.056757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c349899-aaa8-4331-a124-a7599d5351de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:20.057068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c349899-aaa8-4331-a124-a7599d5351de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:20.058506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c349899-aaa8-4331-a124-a7599d5351de None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:20.079326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 318/632] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:46:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:20.085299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28fce-59fe-4016-8d49-6646c404dc47 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.094995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25308b-637d-41c0-8331-4962c85ebdb0 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.095562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25308b-637d-41c0-8331-4962c85ebdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.096050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25308b-637d-41c0-8331-4962c85ebdb0 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.099853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 319/633] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:20.103999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c875-f1b3-45bd-b201-7fcc52b6f3af None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.104519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c875-f1b3-45bd-b201-7fcc52b6f3af None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.105029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9c875-f1b3-45bd-b201-7fcc52b6f3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.138245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ccf9c875-f1b3-45bd-b201-7fcc52b6f3af None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:20.140913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ccf9c875-f1b3-45bd-b201-7fcc52b6f3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:20.208265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:20.228231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 320/634] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:46:20.229105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9c875-f1b3-45bd-b201-7fcc52b6f3af None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.229523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28fce-59fe-4016-8d49-6646c404dc47 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.257468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:20.283076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.284572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.285425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.286155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.343978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9c875-f1b3-45bd-b201-7fcc52b6f3af None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.378142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:20.379947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:20.380697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.381377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.381687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.381923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.396878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:20.403304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:20.404226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:20.405097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:20.411840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 321/635] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:20.411840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.468872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:20.508386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 315/636] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:20.509540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.528080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:20.546262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abc92d-7954-4f28-9076-32ced22b78bd None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.584567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.585152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.585435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.585708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.596772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:20.602098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:20.603107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:20.603961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 322/637] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:20.604640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.621838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076539f-b7f5-4796-945d-9f1943d7f418 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.656101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.656693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.657008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.657277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.657938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:20.658234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:20.660163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:20.668243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 316/638] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:20.669047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.727518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468dca2e-68e4-4f8e-a068-b89f0209c581 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.729515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:20.753684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:20.769189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.769997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.770554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.770833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.775919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbb2ae-c125-42ad-81d7-bb72d53cb858 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.783462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:20.790287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:20.791212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 323/639] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:20.797439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.809892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.810516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.810798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.811076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.822225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:20.823694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:20.838654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:20.839596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:20.840495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:20.840495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:20.840495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:20.844270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 317/640] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:20.845147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.906969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57661a52-23cf-498f-af61-c9673c82e74a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.946420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:20.947030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:20.947303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:20.947570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:20.948227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:20.948522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:20.950113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:20.956478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7b01b-7424-4d73-8761-909bc7c97fae None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:20.962685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 324/641] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:20.962685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:20.989670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:20.994079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.001008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.001646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.001925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.002201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.013140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.014613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:21.029422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:21.030458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:21.031373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:21.031373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:21.031373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:21.031373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 318/642] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:21.031373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.081073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08127e2e-1bf9-4947-a2b2-7aba877acede None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.116926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.117812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.118097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.118373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.119038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:21.119340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.121105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.129305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 325/643] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:21.130361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.144372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f710aa-c03a-42f6-a45b-b2a39289cab6 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.180742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.181661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.181947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.182218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.182859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:21.183162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.185006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.209932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 319/644] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:21.210588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.218532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.237308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084ec04-bbd8-4e9d-ad92-2a08a134f4e0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.255532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.273631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.274711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.275174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.275448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.287122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.288583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:21.309461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:21.310438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:21.312069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:21.312069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:21.312069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:21.312069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 326/645] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:21.312069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.328029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513a7fe-3793-4c8b-b458-48cbb8a1634f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.362428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.363034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.363330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.363573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.364222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:21.364520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.365946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.377074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 320/646] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:21.378451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.428565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa65770-1271-4fe5-a575-bbcfc972b24b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.472800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.473437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.474013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.474285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.478137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7b72c-d8a3-4d61-9adc-3e67d6b35ad9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.479671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.483248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.488981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.490586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:21.506685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:21.507654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:21.508530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:21.508530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:21.508530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:21.508530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 327/647] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:21.509201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.519296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.519930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.520205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.520475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.527331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.528805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:21.540542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:21.541467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:21.542337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 321/648] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:21.542805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.630912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f45d1-f3f3-4147-b3e3-fcabf8814045 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.663906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbaee6-b5b4-4674-ad05-1ed2a28b8688 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.672112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.672752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.673025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.673296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.673939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:21.674845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.676311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.688182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 328/649] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:21.689272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.701699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.702683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.702959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.703228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.711286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.712759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:21.726052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:21.726982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:21.727913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 322/650] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:21.728540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.748148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.754783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.801330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1c02b-30a3-4064-b64d-5e1e786d0ab5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.838405 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bbd91-f111-4472-ae58-61097709f729 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.841400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.842020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.842300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.842571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.843218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:21.843514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.844924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.858979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 329/651] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:21.864488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.877600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:21.878845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:21.879302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:21.879901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:21.881249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:21.882018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:21.884079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:21.901387 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 323/652] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:21.902537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:21.972884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c896e9-5d1b-430e-8ac5-4c01fc251837 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:21.983810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:21.991516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.007891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.008515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.008814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.009084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.010153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972c69c-6445-46d0-bb7f-599a09297ff5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.017095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.019076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:22.030225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:22.031426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 330/653] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:22.032511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.048997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.049635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.049994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.050276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.051021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:22.051332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.052801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.070042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 324/654] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 15:46:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:22.070842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.136363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827301c5-ebcd-43a7-a6e2-463322deb24d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.168901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227cee0-2301-44e8-acca-052d16c3723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.169456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6c558-8617-48cf-8abf-1d7b67277ec2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.169564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227cee0-2301-44e8-acca-052d16c3723b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.169779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227cee0-2301-44e8-acca-052d16c3723b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.170044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227cee0-2301-44e8-acca-052d16c3723b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.178171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227cee0-2301-44e8-acca-052d16c3723b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.179615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227cee0-2301-44e8-acca-052d16c3723b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-8227cee0-2301-44e8-acca-052d16c3723b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:22.190565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:22.191480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:22.192434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 331/655] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:22.193097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c0fd6-3eb6-478d-b1e8-b45359b8922d None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.193222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c0fd6-3eb6-478d-b1e8-b45359b8922d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.197907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c0fd6-3eb6-478d-b1e8-b45359b8922d None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.197907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 332/656] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:22.197907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c0fd6-3eb6-478d-b1e8-b45359b8922d None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.208317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.209032 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.209309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.209573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.210300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=ed959e4bdaf44132b05ccaec68c41172) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:22.210601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.212038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.218000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 325/657] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/role_assignments?user.id=ed959e4bdaf44132b05ccaec68c41172&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:22.218992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.225365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.234449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.331519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c0fd6-3eb6-478d-b1e8-b45359b8922d None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.338840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309a2a4-089f-4d3d-a619-15aee1293af7 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.373759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.374709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.375054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.375324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.376062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:22.376369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.377897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.378799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.379427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.380028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.380303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.397293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 333/658] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:22.398054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.463682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.465091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.469424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.497261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.511076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6f54-ada2-40dd-a7d4-e166af34e997 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.548873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312facaf-26cb-44f7-9531-342aa9e11c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.549521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312facaf-26cb-44f7-9531-342aa9e11c7d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.549797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312facaf-26cb-44f7-9531-342aa9e11c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.550065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312facaf-26cb-44f7-9531-342aa9e11c7d None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.550792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312facaf-26cb-44f7-9531-342aa9e11c7d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:22.551093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312facaf-26cb-44f7-9531-342aa9e11c7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.552518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312facaf-26cb-44f7-9531-342aa9e11c7d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.571390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 334/659] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:22.572603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a047d1d-a3c5-4629-a2ca-8359de2478dc None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.572968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a047d1d-a3c5-4629-a2ca-8359de2478dc None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.573247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a047d1d-a3c5-4629-a2ca-8359de2478dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.584060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 326/660] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:22.588578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.593560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7a047d1d-a3c5-4629-a2ca-8359de2478dc None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:22.594743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7a047d1d-a3c5-4629-a2ca-8359de2478dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:22.676824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 335/661] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:22.676824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a047d1d-a3c5-4629-a2ca-8359de2478dc None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.716237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c735b4a-b052-4442-9dba-6d9a25f4f375 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.719257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.747771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.764410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.765223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.765596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.765965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.777789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a047d1d-a3c5-4629-a2ca-8359de2478dc None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:22.815165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:22.815813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:22.816091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:22.816370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:22.860580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.863117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.890762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:22.892327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:22.973029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:22.978003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 327/662] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:22.978003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:22.983668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.005073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 336/663] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:23.006281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.092216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddc6e8-a1e6-4084-b7b4-cf548efc3a37 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.110417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe88401-2239-48fa-981a-c64a62511229 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.129298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.130253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.130713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.131164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.143165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.145146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.145485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.145769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.146042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.146309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.155974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:23.161304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:23.162243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 337/664] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:23.163180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:23.165087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:23.166001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:23.166868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:23.166868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.166868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:23.169091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 328/665] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:23.170074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.204116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.239304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.280969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30718e06-4c17-45db-ab8e-6efd11260374 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.286767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b243c-feca-4fe3-b33c-dde913aaacd1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.324139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.324876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.325216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.325551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.327642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.328435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.328542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.328837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.336832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.338371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.340236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.341916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:23.353826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:23.354751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:23.355631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:23.355631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.355631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:23.355631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 338/666] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:23.355631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:23.359800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:23.360923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:23.362065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:23.362065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.362065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:23.366994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 329/667] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:23.367890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.454303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.468445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94638161-d856-4cdd-b94b-6cdca2ddeeda None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.478601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6efd-a89a-4dca-a292-8cded73b4943 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.487872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.506878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.507402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.507730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.507942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.508599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:23.508924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.510283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.513458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.514064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.514340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.514612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.515260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:23.515572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.516960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.519541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 339/668] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:23.520525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.534629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 330/669] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:23.535463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.634340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff771a-cda4-43b3-ab42-07b7549e37a9 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.642970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f703607-b6ee-4e1a-9521-f4d4a5318b96 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.676523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.677159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.677433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.677700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.678336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:23.678633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.680008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.684127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.684749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.685041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.685314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.685960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:23.686264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.688135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.696421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 340/670] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:23.696421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.702345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 331/671] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:23.707311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.709975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.738995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.808831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e56dfa-5b7d-4f67-9859-7308e80b0f37 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.816169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9871c7-48f7-4fdb-8fdc-f570a790059d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.845175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.845764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.846638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.846915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.853015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:23.853971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:23.854428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:23.854869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:23.857225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.858680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.862887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:23.864520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:23.873671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:23.874594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:23.875550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:23.875550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:23.875550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:23.875550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 341/672] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:23.875550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:23.879498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:23.880410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 332/673] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:23.882355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:23.952690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.975001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:23.990906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24c73-73d3-48c1-8c26-bbe6371a964b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:23.997203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bec775-9e69-458e-b766-9761d982d4d4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.028763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.029401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.029684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.029957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.033288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.034209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.034699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.035177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.037094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.038551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.042730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.044445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:24.050818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:24.051698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:24.052580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 333/674] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:24.053086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:24.062354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:24.063355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:24.064409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:24.070892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 342/675] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:24.071886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.155395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6e462-be4c-4573-81c0-81185f53a23d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.181089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648678b-1194-487b-bbe2-627129bf4bcf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.189123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.189759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.190570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.190845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.191585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.191890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.193309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.206099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 334/676] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.206937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.210909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:24.219532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:24.225213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.225823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.226103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.226374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.227067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.227371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.228819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.241791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 343/677] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.241791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.320963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514beef-b59d-47c5-b6ba-66f1cbbff46e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.357198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.357826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.358106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.358374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.359110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.359402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.360934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.362585 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d032b-3c91-4f5d-9a82-dc90c141f758 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.374771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 335/678] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.375647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.407117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.407736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.408013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.408277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.409086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.409438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.410808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.427336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 344/679] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.427336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.458063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:24.478686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:24.493760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc59c5-0894-447f-a36e-86aadef194ff None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.528286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.528933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.529215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.529485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.536013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefc94f-ba70-429d-b40a-bbcfa9716404 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.536918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.538383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:24.549378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:24.550479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 336/680] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:24.551425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.573213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.573751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.574026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.574304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.575027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=4396302927da47a3a80eae4c50fdb266) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.575331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.576783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.583126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 345/681] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/role_assignments?user.id=4396302927da47a3a80eae4c50fdb266&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.583927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.664940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd87eb-0024-4946-9ba6-6ddda8824af2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.695742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:24.705713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d36d6-f5a3-4fa4-b9f4-fb30bb3011af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.706448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d36d6-f5a3-4fa4-b9f4-fb30bb3011af None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.706725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d36d6-f5a3-4fa4-b9f4-fb30bb3011af None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.706996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d36d6-f5a3-4fa4-b9f4-fb30bb3011af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.707734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d36d6-f5a3-4fa4-b9f4-fb30bb3011af None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=2d007a3d205345aebcc36a36d039da2d) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.708041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d36d6-f5a3-4fa4-b9f4-fb30bb3011af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.709548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d36d6-f5a3-4fa4-b9f4-fb30bb3011af None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.709836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4e305-3944-4862-aaed-2eace79eb4c8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.722126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 337/682] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/role_assignments?user.id=2d007a3d205345aebcc36a36d039da2d&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.722126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888acc80-d008-4f39-bdac-b2b74e5810ef None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.722126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888acc80-d008-4f39-bdac-b2b74e5810ef None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.722576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:24.722795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888acc80-d008-4f39-bdac-b2b74e5810ef None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.722795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 338/683] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:24.722795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888acc80-d008-4f39-bdac-b2b74e5810ef None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:24.772220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10739881-c289-4c26-934e-07a023f2fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.773221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739881-c289-4c26-934e-07a023f2fed9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.773610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739881-c289-4c26-934e-07a023f2fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.773882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10739881-c289-4c26-934e-07a023f2fed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.774626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10739881-c289-4c26-934e-07a023f2fed9 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=022516f130ec4ad6a9b7c46b417a5690) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.774930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10739881-c289-4c26-934e-07a023f2fed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.776571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10739881-c289-4c26-934e-07a023f2fed9 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.786895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 346/684] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/role_assignments?user.id=022516f130ec4ad6a9b7c46b417a5690&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.787523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b2dd9-844c-4fe6-af4a-15e9fbc58839 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.787818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b2dd9-844c-4fe6-af4a-15e9fbc58839 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.788470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b2dd9-844c-4fe6-af4a-15e9fbc58839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.814065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-2f7b2dd9-844c-4fe6-af4a-15e9fbc58839 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:24.815575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2f7b2dd9-844c-4fe6-af4a-15e9fbc58839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:24.865293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888acc80-d008-4f39-bdac-b2b74e5810ef None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:24.913133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:24.913858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:24.914134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:24.914401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:24.914847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 347/685] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:24.915281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:24.915523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:24.917064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:24.939234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 339/686] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:24.945194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b2dd9-844c-4fe6-af4a-15e9fbc58839 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:25.053104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:25.056508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b2dd9-844c-4fe6-af4a-15e9fbc58839 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:25.092919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:25.093578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:25.093859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:25.094126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:25.167039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:25.168415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:25.208780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:25.282162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:25.309868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:25.325046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 348/687] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:25.329800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:25.398427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302d25-fad3-4f96-96ae-851ab19d27b4 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:25.470089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:25.471571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:25.472164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:25.472590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:25.493213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc143b-7dd2-44f9-a6f8-0ee1e92e4dca None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:25.494000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:25.500439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:25.501538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:25.502667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:25.515249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 340/688] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:25.519516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:25.570964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:25.572021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:25.573242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:25.574285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:25.602813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:25.604277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:25.632898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:25.634147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:25.635448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:25.635448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:25.635448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:25.635448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 349/689] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:25.639411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:25.666979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ab1ea-d675-4702-a1e8-4c4a5ac9af13 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:25.680633 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:25.736201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:25.736979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:25.737391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:25.737668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:25.738326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:25.738621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:25.740005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:25.757265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:25.759117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 341/690] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:25.764974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:25.845399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad596f-2abe-44da-a369-4f9d7cb1935a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:25.882091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:25.882730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:25.883046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:25.883322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:25.883985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:25.884284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:25.886313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:25.900599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 350/691] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:25.905969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:25.974504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c7f85-8bc2-4899-ae86-4ccbc469cb8e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:26.013728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72895e-2a0d-425a-97cc-3f38b89ad12a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:26.049261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:26.053405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:26.054113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:26.054397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:26.054667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:26.062154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:26.063679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:26.075970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:26.076965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:26.078268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 351/692] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:26.083052 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:26.093443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:26.094099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:26.094378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:26.094646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:26.133589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:26.135086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:26.166407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:26.167418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:26.168320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:26.168320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:26.168320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:26.176929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 342/693] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:26.181725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:26.191681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f7580-053f-4f4b-8523-8aacb5b0c5d9 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:26.228353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:26.228991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:26.229273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:26.229546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:26.230236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:26.230540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:26.231986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:26.244589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 352/694] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:26.250795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:26.274144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:26.297673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:26.415084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeb743-f304-4420-90f3-f15c166f9e06 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:26.463979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a01b06-47eb-4034-84dd-53f192305b65 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:26.495426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:26.496041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:26.496327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:26.496598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:26.497360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=7701e2653e4b4ca7a842887c0526862f) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:26.497685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:26.499098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:26.517291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 353/695] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:26 2022] GET /identity/v3/role_assignments?user.id=7701e2653e4b4ca7a842887c0526862f&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:26.538679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:26.539235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:26.539513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:26.539784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:26.540461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:26.540793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:26.542244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:26.563821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 343/696] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:26.569686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:26.675009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:26.681500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d580a9c-0029-48cc-9b77-195f639b0459 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:26.719365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:26.720018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:26.720296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:26.720567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:26.729996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:26.731526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:26.744952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:26.746001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:26.747200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 354/697] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:46:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:26.758948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:26.864124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:26.882413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa240f-305c-4ab6-846b-235233320eef None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:26.925304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d49e9-1608-4933-a431-dda0668ad97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:26.925922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d49e9-1608-4933-a431-dda0668ad97f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:26.926199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d49e9-1608-4933-a431-dda0668ad97f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:26.926466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d49e9-1608-4933-a431-dda0668ad97f None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:26.927203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d49e9-1608-4933-a431-dda0668ad97f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:26.928373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d49e9-1608-4933-a431-dda0668ad97f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:26.928983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d49e9-1608-4933-a431-dda0668ad97f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:26.948262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 344/698] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:46:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:26.953709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:27.067261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949f4f6-09a7-43d0-a6d8-871f33732b03 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:27.105391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c6d9a-a747-4c66-997a-30bb279bad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:27.106068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c6d9a-a747-4c66-997a-30bb279bad0b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.106351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c6d9a-a747-4c66-997a-30bb279bad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.106619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c6d9a-a747-4c66-997a-30bb279bad0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.134205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:27.171747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705edebe-0f1b-4896-80f1-48060dcf159a None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.172050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705edebe-0f1b-4896-80f1-48060dcf159a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.172298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705edebe-0f1b-4896-80f1-48060dcf159a None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.176477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 345/699] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:27.180089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c6d9a-a747-4c66-997a-30bb279bad0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:27.181743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037468ce-3e3f-4d96-a546-61a0d4099f5b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.182033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037468ce-3e3f-4d96-a546-61a0d4099f5b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.182311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037468ce-3e3f-4d96-a546-61a0d4099f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.182633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c6d9a-a747-4c66-997a-30bb279bad0b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:27.209567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-037468ce-3e3f-4d96-a546-61a0d4099f5b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:27.211716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-037468ce-3e3f-4d96-a546-61a0d4099f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:27.324792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 346/700] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:27.339615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 355/701] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:27.345304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037468ce-3e3f-4d96-a546-61a0d4099f5b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:27.429708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:27.463599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037468ce-3e3f-4d96-a546-61a0d4099f5b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:27.477977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57947a04-8202-4974-b995-1660b5f0a61a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.478276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57947a04-8202-4974-b995-1660b5f0a61a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.478526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57947a04-8202-4974-b995-1660b5f0a61a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.479859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 356/702] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:27.493610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1434c50-0b18-4478-b818-5857bea93c69 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.493916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1434c50-0b18-4478-b818-5857bea93c69 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.494228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1434c50-0b18-4478-b818-5857bea93c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.506558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:27.507194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.507477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.507745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.515072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f1434c50-0b18-4478-b818-5857bea93c69 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:27.516492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f1434c50-0b18-4478-b818-5857bea93c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:27.519559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:27.521062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:27.536937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:27.537991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:27.538852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:27.538852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:27.538852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:27.538852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 347/703] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:27.543766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:27.586834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:27.614081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 357/704] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:27.666167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1434c50-0b18-4478-b818-5857bea93c69 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:27.671306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1c8dd-aae2-4ad3-8c8c-10623e603ae2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:27.705308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:27.733532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6d59a-20ac-4659-bc6e-76a4d933db7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:27.734152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d59a-20ac-4659-bc6e-76a4d933db7b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.734431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d59a-20ac-4659-bc6e-76a4d933db7b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.734699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d59a-20ac-4659-bc6e-76a4d933db7b None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.735360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6d59a-20ac-4659-bc6e-76a4d933db7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:27.735659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6d59a-20ac-4659-bc6e-76a4d933db7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:27.738066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6d59a-20ac-4659-bc6e-76a4d933db7b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:27.755613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 348/705] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:27.757980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f5280-3aeb-440c-bfdd-216c1a2d3f90 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.758461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f5280-3aeb-440c-bfdd-216c1a2d3f90 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.758926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f5280-3aeb-440c-bfdd-216c1a2d3f90 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.764953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 349/706] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:27.764953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f5280-3aeb-440c-bfdd-216c1a2d3f90 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:27.821154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1434c50-0b18-4478-b818-5857bea93c69 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:27.867734 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049fcf3-ab4d-4f50-a52b-5baac7cc15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:27.868468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049fcf3-ab4d-4f50-a52b-5baac7cc15fd None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.868848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049fcf3-ab4d-4f50-a52b-5baac7cc15fd None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.869179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049fcf3-ab4d-4f50-a52b-5baac7cc15fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.870120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049fcf3-ab4d-4f50-a52b-5baac7cc15fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:27.871572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049fcf3-ab4d-4f50-a52b-5baac7cc15fd None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-5049fcf3-ab4d-4f50-a52b-5baac7cc15fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:27.886006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:27.887055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:27.887898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 358/707] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Oct 3 15:46:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:27.888808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c9c5-fbcd-4cd3-8506-dcd901c50c6c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.889546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c9c5-fbcd-4cd3-8506-dcd901c50c6c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.889921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13c9c5-fbcd-4cd3-8506-dcd901c50c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.900495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f5280-3aeb-440c-bfdd-216c1a2d3f90 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:27.921505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bb13c9c5-fbcd-4cd3-8506-dcd901c50c6c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:27.922999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bb13c9c5-fbcd-4cd3-8506-dcd901c50c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:27.947173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:27.947803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:27.948076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:27.948342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:27.955100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:27.957261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:27.968502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:27.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:27.970500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 350/708] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:27.971681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:27.981666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:28.004442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 359/709] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:28.005529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c9c5-fbcd-4cd3-8506-dcd901c50c6c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.031673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:28.101567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40093da2-7736-40d5-9d42-944339729eaa None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.125361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13c9c5-fbcd-4cd3-8506-dcd901c50c6c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.138398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.139119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.139470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.139815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.140524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:28.141060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.142589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.152864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 351/710] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 15:46:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:28.154252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.166884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.167945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.168226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.168488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] PATH_INFO: `/v3/services/network` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.169111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.170616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:28.181407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:28.182451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:28.188225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 360/711] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:28.189080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.237976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:28.285452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe22cd4-6f50-4cef-8821-f996d24e3f8a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.296146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:28.312150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb5ffb-e44e-49c3-9563-b53e5375dfff None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.327119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90572440-516d-4300-b46a-405524139c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.328001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90572440-516d-4300-b46a-405524139c8d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.328484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90572440-516d-4300-b46a-405524139c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.329108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90572440-516d-4300-b46a-405524139c8d None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.329929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90572440-516d-4300-b46a-405524139c8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:28.330943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90572440-516d-4300-b46a-405524139c8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.332487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90572440-516d-4300-b46a-405524139c8d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.350152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 352/712] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:28.351010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90572440-516d-4300-b46a-405524139c8d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.351988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.352588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.352879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.353144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.353761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:28.354059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.355548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.368769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 361/713] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:28.368769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.481340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab8fd1-3323-402c-818d-e226eaf7a26d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.485455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90572440-516d-4300-b46a-405524139c8d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.509359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:28.529291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.530406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.530692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.530966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.536409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.537214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.537592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.537961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.538712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:28.539140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.541249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.543940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.546630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.547611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 353/714] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:28.549230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:28.573088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:28.574282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:28.575345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:28.575345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:28.575345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:28.575345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 362/715] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:28.575345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.593599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:28.677765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e4032-3d13-438d-ae36-1eee8d80d9b1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.699653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564f091-1c27-440f-b130-f9d3d1ceed6e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.725336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.726875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.727399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.728109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.729352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=ed959e4bdaf44132b05ccaec68c41172) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:28.729671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.731136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.743265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.744253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.744751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.745241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.746081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] RBAC: Adding query filter params (type=network) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:28.746603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.748205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.748858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 354/716] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/role_assignments?user.id=ed959e4bdaf44132b05ccaec68c41172&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:28.749864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:28.755006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 363/717] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:28.878986 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:28.885791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2933e64-2835-40da-b009-1bfbbdc62527 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:28.941099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f2f85-2c3b-4c57-8677-8569d4ad49c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:28.941737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f2f85-2c3b-4c57-8677-8569d4ad49c8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:28.942015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f2f85-2c3b-4c57-8677-8569d4ad49c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:28.942286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f2f85-2c3b-4c57-8677-8569d4ad49c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:28.942969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f2f85-2c3b-4c57-8677-8569d4ad49c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:28.943276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f2f85-2c3b-4c57-8677-8569d4ad49c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:28.944744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f2f85-2c3b-4c57-8677-8569d4ad49c8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:28.962307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 355/718] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:28.967828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:29.017014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:29.100460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af426c56-ee5b-462d-8856-8bf4afe71fef None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:29.136179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8159486-843f-4edc-ba87-6293bb9ee597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:29.137556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159486-843f-4edc-ba87-6293bb9ee597 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:29.138030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159486-843f-4edc-ba87-6293bb9ee597 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:29.138513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8159486-843f-4edc-ba87-6293bb9ee597 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:29.139360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8159486-843f-4edc-ba87-6293bb9ee597 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:29.141020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8159486-843f-4edc-ba87-6293bb9ee597 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:29.166558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 364/719] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:46:28 2022] POST /identity/v3/users => generated 254 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:29.563108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774fecb-c7d8-441f-9217-9b04e7896d15 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:29.563476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774fecb-c7d8-441f-9217-9b04e7896d15 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:29.563653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774fecb-c7d8-441f-9217-9b04e7896d15 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:29.565477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 356/720] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:29.571186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8d4a-70cb-4c8f-b4fa-6db8fffa983a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:29.571474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8d4a-70cb-4c8f-b4fa-6db8fffa983a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:29.571748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8d4a-70cb-4c8f-b4fa-6db8fffa983a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:29.591827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-899d8d4a-70cb-4c8f-b4fa-6db8fffa983a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:29.593040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-899d8d4a-70cb-4c8f-b4fa-6db8fffa983a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:29.675953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 365/721] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:46:29.990599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774fecb-c7d8-441f-9217-9b04e7896d15 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:30.119572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774fecb-c7d8-441f-9217-9b04e7896d15 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:30.163658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:30.164339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:30.164621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:30.164917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:30.180682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:30.185524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:30.186478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:30.187303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:30.189390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 357/722] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:30.194469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d8d4a-70cb-4c8f-b4fa-6db8fffa983a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:30.207363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:30.322976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d8d4a-70cb-4c8f-b4fa-6db8fffa983a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:30.365055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:30.365667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:30.365947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:30.366215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:30.366882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:30.367244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:30.368617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:30.378345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 366/723] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:30.383705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:30.411167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:30.546254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e6601-5aab-4805-a217-163e483eb728 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:30.605797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:30.606342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:30.606616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:30.606876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:30.622282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:30.623817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:30.652400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:30.653806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:30.655418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:30.655418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:30.655418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:30.655418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 358/724] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:30.665295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:30.798036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:30.837260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce46e0e-ee5f-45e2-bf3a-10b10d0e720b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:30.878470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36255d1e-db62-4388-ad06-28eecc2e46a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:30.879074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255d1e-db62-4388-ad06-28eecc2e46a8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:30.879356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255d1e-db62-4388-ad06-28eecc2e46a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:30.879634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36255d1e-db62-4388-ad06-28eecc2e46a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:30.880940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255d1e-db62-4388-ad06-28eecc2e46a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:30.881347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255d1e-db62-4388-ad06-28eecc2e46a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:30.883433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36255d1e-db62-4388-ad06-28eecc2e46a8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:30.897545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 367/725] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:30.906167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:31.019392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38e64f-44c8-47ab-b272-c34f9bd7861a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.020006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38e64f-44c8-47ab-b272-c34f9bd7861a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.020546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38e64f-44c8-47ab-b272-c34f9bd7861a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.024756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 368/726] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:31.025379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:31.028873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7454c-2772-42d4-8534-fb3a9bb3c43f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.029163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7454c-2772-42d4-8534-fb3a9bb3c43f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.029435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7454c-2772-42d4-8534-fb3a9bb3c43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.051372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0be7454c-2772-42d4-8534-fb3a9bb3c43f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:31.053478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0be7454c-2772-42d4-8534-fb3a9bb3c43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:31.087593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca236-9654-4210-9eff-d903c9962dc4 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:31.168796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:31.169364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.169640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.169915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.192517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:31.195191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:31.210931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:31.211927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:31.212878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 359/727] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:31.215591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 369/728] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 15:46:31.223955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:31.363164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351d5f2-c04f-4c09-805d-8b0e71d5a87a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:31.406050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:31.414761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:31.415374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.415646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.415909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.416648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:31.416973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:31.418421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:31.448251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 360/729] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 15:46:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:31.454010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7454c-2772-42d4-8534-fb3a9bb3c43f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:31.537065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:31.590893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7454c-2772-42d4-8534-fb3a9bb3c43f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:31.636786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:31.641653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca911a5c-043a-4025-bda4-c5380d76609f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:31.642278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca911a5c-043a-4025-bda4-c5380d76609f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.642591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca911a5c-043a-4025-bda4-c5380d76609f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.642856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca911a5c-043a-4025-bda4-c5380d76609f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.723993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca911a5c-043a-4025-bda4-c5380d76609f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:31.725401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca911a5c-043a-4025-bda4-c5380d76609f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:31.799283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9131a-5495-40ba-9e6a-ef901b56ac7b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:31.802700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:31.835377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc2048-7cb4-4116-b339-f91b808d6560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:31.835985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc2048-7cb4-4116-b339-f91b808d6560 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.836259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc2048-7cb4-4116-b339-f91b808d6560 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.836525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc2048-7cb4-4116-b339-f91b808d6560 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.837220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc2048-7cb4-4116-b339-f91b808d6560 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:31.838592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc2048-7cb4-4116-b339-f91b808d6560 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:31.851512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 370/730] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 15:46:31.852920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b74b-f158-4d0a-9b03-fc88aea3e69a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.853383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b74b-f158-4d0a-9b03-fc88aea3e69a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.853756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b74b-f158-4d0a-9b03-fc88aea3e69a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.860998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 361/731] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:46:31 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:46:31.860998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425611-5c64-4067-91eb-3faf40cc92f9 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.860998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425611-5c64-4067-91eb-3faf40cc92f9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.860998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425611-5c64-4067-91eb-3faf40cc92f9 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.861944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 371/732] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:31.861944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b74b-f158-4d0a-9b03-fc88aea3e69a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:31.864581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 362/733] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:31.864581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42361df-8637-4f35-b876-27935d681a32 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.864846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42361df-8637-4f35-b876-27935d681a32 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.864930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42361df-8637-4f35-b876-27935d681a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.889258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c42361df-8637-4f35-b876-27935d681a32 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:31.890836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c42361df-8637-4f35-b876-27935d681a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:31.945185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:31.989338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 363/734] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:31.991137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77db0e2-55a8-4c2a-9cbd-062060226966 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:31.991572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77db0e2-55a8-4c2a-9cbd-062060226966 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:31.991983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77db0e2-55a8-4c2a-9cbd-062060226966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:31.995749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b74b-f158-4d0a-9b03-fc88aea3e69a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:32.021232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d77db0e2-55a8-4c2a-9cbd-062060226966 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:32.022767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d77db0e2-55a8-4c2a-9cbd-062060226966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:32.051212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:32.051212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.051212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.051212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.058909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:32.060395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:32.121067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 364/735] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:32.194123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:32.195134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:32.196099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:32.196099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:32.196099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:32.196099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 372/736] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:32.202287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77db0e2-55a8-4c2a-9cbd-062060226966 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:32.332531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77db0e2-55a8-4c2a-9cbd-062060226966 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:32.334121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:32.352661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:32.371017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:32.371646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.371921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.372190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.372900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:32.373233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:32.374895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:32.375926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:32.396303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 365/737] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:32.403068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:32.513110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5bb26-debb-4672-bea7-87725a46e748 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:32.539852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849c7d-a38b-4119-abaa-523f12cf28ab None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:32.554476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:32.555194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.555384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.555630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.563503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:32.565731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:32.575938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe9647-bec9-4a5d-bafc-3b01158fdcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:32.576674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9647-bec9-4a5d-bafc-3b01158fdcbd None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.577121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9647-bec9-4a5d-bafc-3b01158fdcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.577514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9647-bec9-4a5d-bafc-3b01158fdcbd None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:32.578123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:32.579681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:32.582183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:32.582980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe9647-bec9-4a5d-bafc-3b01158fdcbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:32.583116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 366/738] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:32.583116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:32.583343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe9647-bec9-4a5d-bafc-3b01158fdcbd None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:32.608759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 373/739] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:46:32 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:46:32.610505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:32.616422 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96b555-3a8b-4601-83f5-73b663ff11ae None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.617257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96b555-3a8b-4601-83f5-73b663ff11ae None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.617880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96b555-3a8b-4601-83f5-73b663ff11ae None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.621688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 374/740] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:32.623603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96b555-3a8b-4601-83f5-73b663ff11ae None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:32.725169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:32.725980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca81f8-a9c3-4537-9443-c0c0207b0dcc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:32.779272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96b555-3a8b-4601-83f5-73b663ff11ae None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:32.780526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdbc778-3741-4dcb-88ad-94550a608fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:32.781269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc778-3741-4dcb-88ad-94550a608fce None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.781902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc778-3741-4dcb-88ad-94550a608fce None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.782181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdbc778-3741-4dcb-88ad-94550a608fce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.797721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdbc778-3741-4dcb-88ad-94550a608fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-5fdbc778-3741-4dcb-88ad-94550a608fce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:32.805638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:32.806857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:32.807876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:32.811836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 367/741] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:32.813634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658984f0-9223-4f49-acac-1e9c2b731392 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.813933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658984f0-9223-4f49-acac-1e9c2b731392 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.814208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658984f0-9223-4f49-acac-1e9c2b731392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.833665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:32.834377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:32.834666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:32.834939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:32.835646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:32.835950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:32.836240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-658984f0-9223-4f49-acac-1e9c2b731392 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:32.837461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:32.837804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-658984f0-9223-4f49-acac-1e9c2b731392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:32.852887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 375/742] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:32.852887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:32.919083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 368/743] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:32.920367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658984f0-9223-4f49-acac-1e9c2b731392 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:32.931931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:32.978569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ba4b9-6a0c-4c6d-9a50-0f9e43ed0379 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:33.039945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:33.051359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:33.051990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:33.052266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:33.052536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:33.053240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:33.053544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:33.058395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:33.077104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 376/744] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:33.084004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:33.099613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658984f0-9223-4f49-acac-1e9c2b731392 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:33.172923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:33.173706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:33.173910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:33.174182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:33.174927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=7701e2653e4b4ca7a842887c0526862f) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:33.175240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:33.176692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:33.196891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 369/745] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:32 2022] GET /identity/v3/role_assignments?user.id=7701e2653e4b4ca7a842887c0526862f&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:33.230879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:33.278008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed8fd7-7ecd-4f9e-ae81-799983e3af97 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:33.352213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:33.352806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:33.353337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:33.353337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:33.361100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:33.372099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:33.376772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:33.401297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:33.402289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:33.403179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:33.403179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:33.403179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:33.404689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acffa4-1bd3-491d-8517-358927bc9ac8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:33.408998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 377/746] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:33.411967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:33.461530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:33.462641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:33.462917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:33.463337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:33.480004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:33.488053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:33.489080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:33.490743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 370/747] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:33.495237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:33.504851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:33.602967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4f4d5-3045-49d3-bbd1-66ccde29d08d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:33.640596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:33.642143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:33.642452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:33.642722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:33.643372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:33.643665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:33.645186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:33.675207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 378/748] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:33.678439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03392825-095c-464c-a5f6-aa80a7414182 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:33.680764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:33.739953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:33.752580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:33.753213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:33.753484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:33.753744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:33.754411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:33.754707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:33.756241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:33.770167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 371/749] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:33.780046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:33.856172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c40cc-b1d6-495b-9410-414b01e46689 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:33.869069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:33.895594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:33.896293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:33.896615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:33.896934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:33.905014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:33.906975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:33.919934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:33.921535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:33.922467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 379/750] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:46:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:33.929407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:33.993690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4511eed-9a56-46a2-99fc-b431cc161988 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.035040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487edbe-2898-48f2-b624-b8c22a75d67d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.062825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.063465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.063745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.064013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.071880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.072601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.073000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.073378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.074201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:34.074596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.076328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.087133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.088611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.096560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 380/751] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:46:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:34.102556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:34.108121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:34.109074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:34.109955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:34.109955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:34.109955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:34.109955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 372/752] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:34.126235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.142444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:34.217238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d974d7-6cde-4211-89ab-2c301772e2aa None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.217952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:34.256523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.257191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.257462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.257731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.268846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.271071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:34.289226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:34.291400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:34.293571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:34.293571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:34.293571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:34.293948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975928f-c0a3-47db-b7bc-451cdfa8fff5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.294102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 381/753] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:34.300747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.368527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.369417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.369850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.370273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.371177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:34.371700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.373262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.395254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:34.403252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 373/754] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:34.411438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.421538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d0f9-275d-4c0f-9353-5a3eadc84686 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.462483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.463134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.463409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.463675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.464338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:34.464639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.466217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.481254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 382/755] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:34.487279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.602195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52932c40-f6af-4fa3-b845-4c26d8dd22ba None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.608685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302fd47-d572-454b-aa33-bbce2c4a73df None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.617946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:34.650877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba1ac5-5cea-4094-9dd2-7f632827ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.651416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba1ac5-5cea-4094-9dd2-7f632827ae3c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.651690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba1ac5-5cea-4094-9dd2-7f632827ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.651957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba1ac5-5cea-4094-9dd2-7f632827ae3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.654223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:34.655997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767dd0c-bde6-479a-aa49-3a1c07e4a74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.656907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767dd0c-bde6-479a-aa49-3a1c07e4a74d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.657403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767dd0c-bde6-479a-aa49-3a1c07e4a74d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.657889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767dd0c-bde6-479a-aa49-3a1c07e4a74d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.662024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba1ac5-5cea-4094-9dd2-7f632827ae3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.664118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba1ac5-5cea-4094-9dd2-7f632827ae3c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.668062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767dd0c-bde6-479a-aa49-3a1c07e4a74d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.669761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767dd0c-bde6-479a-aa49-3a1c07e4a74d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-b4ba1ac5-5cea-4094-9dd2-7f632827ae3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:34.677679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:34.678659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:34.680097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:34.683659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 383/756] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:34.685094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2a0a3-5c60-445e-a29b-1a1eb0a97359 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.685374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2a0a3-5c60-445e-a29b-1a1eb0a97359 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.685646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2a0a3-5c60-445e-a29b-1a1eb0a97359 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.687183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 384/757] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:34.687879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2a0a3-5c60-445e-a29b-1a1eb0a97359 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-a767dd0c-bde6-479a-aa49-3a1c07e4a74d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:34.692009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:34.692960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:34.693915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:34.696396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 374/758] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:46:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:34.697916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1a893-4028-4de4-ba74-fe0b9530877c None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.698418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1a893-4028-4de4-ba74-fe0b9530877c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.698887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1a893-4028-4de4-ba74-fe0b9530877c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.721960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-23b1a893-4028-4de4-ba74-fe0b9530877c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:34.723820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-23b1a893-4028-4de4-ba74-fe0b9530877c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:34.797936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 375/759] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:34.800490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1a893-4028-4de4-ba74-fe0b9530877c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.827334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2a0a3-5c60-445e-a29b-1a1eb0a97359 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.874379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:34.875024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:34.875532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:34.875865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:34.876566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:34.877023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:34.878993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:34.894670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 385/760] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:34.901092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:34.912018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:34.918959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1a893-4028-4de4-ba74-fe0b9530877c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:34.968971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:35.021281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ba892-6029-48c1-b5aa-e7e829b33759 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:35.057831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:35.058470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.058972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.059333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/ed959e4bdaf44132b05ccaec68c41172/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.061393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=ed959e4bdaf44132b05ccaec68c41172, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:35.062934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:35.079761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 386/761] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:34 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/ed959e4bdaf44132b05ccaec68c41172/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:35.142157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:35.143106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.143601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.144034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.144951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:35.145453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:35.147066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:35.168277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 376/762] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 15:46:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:35.173910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:35.202500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:35.328376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171715d-39ac-4f02-993f-88eb4da4e375 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:35.349071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:35.363512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d38b-d272-4ac7-aef7-781d25c50e93 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:35.377211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:35.378717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.379295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.379764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.398288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c92ef-f877-4625-878b-162c28d59d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:35.398969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c92ef-f877-4625-878b-162c28d59d4c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.399251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c92ef-f877-4625-878b-162c28d59d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.399523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c92ef-f877-4625-878b-162c28d59d4c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.400110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c92ef-f877-4625-878b-162c28d59d4c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:35.401674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c92ef-f877-4625-878b-162c28d59d4c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1c5c92ef-f877-4625-878b-162c28d59d4c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:35.412398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:35.413440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:35.414272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 377/763] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Oct 3 15:46:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:35.415567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e12b0-886d-44c9-beea-3d05f85b115d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.415567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e12b0-886d-44c9-beea-3d05f85b115d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.415792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e12b0-886d-44c9-beea-3d05f85b115d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.421352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 378/764] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:35.422117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e12b0-886d-44c9-beea-3d05f85b115d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:35.449132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:35.470683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:35.473134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:35.531166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e12b0-886d-44c9-beea-3d05f85b115d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:35.566857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb9472-3142-4d04-bd39-f270d7976b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:35.567491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb9472-3142-4d04-bd39-f270d7976b4e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.567768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb9472-3142-4d04-bd39-f270d7976b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.568037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb9472-3142-4d04-bd39-f270d7976b4e None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.568671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb9472-3142-4d04-bd39-f270d7976b4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:35.569140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb9472-3142-4d04-bd39-f270d7976b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:35.570609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb9472-3142-4d04-bd39-f270d7976b4e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:35.580520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 379/765] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 15:46:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:35.581904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce3f4-337c-4a38-86c6-668bab923b39 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.582187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce3f4-337c-4a38-86c6-668bab923b39 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.582457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ce3f4-337c-4a38-86c6-668bab923b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.601632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-201ce3f4-337c-4a38-86c6-668bab923b39 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:35.602473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 387/766] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:46:35.602919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-201ce3f4-337c-4a38-86c6-668bab923b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:35.603729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:35.623073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:35.684900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 380/767] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:35.685735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce3f4-337c-4a38-86c6-668bab923b39 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:35.720566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:35.776653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead23aa-0327-489f-b127-206b440eb88e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:35.846578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:35.847210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.847485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.847759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.848656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:35.848980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:35.850664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:35.864162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 388/768] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 15:46:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:35.869815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:35.892070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ce3f4-337c-4a38-86c6-668bab923b39 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:35.931147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6875af-30f3-4ded-95a5-851fd9f08852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:35.931823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6875af-30f3-4ded-95a5-851fd9f08852 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.932101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6875af-30f3-4ded-95a5-851fd9f08852 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.932388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6875af-30f3-4ded-95a5-851fd9f08852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.944423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6875af-30f3-4ded-95a5-851fd9f08852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:35.945886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6875af-30f3-4ded-95a5-851fd9f08852 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:35.957019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-7b6875af-30f3-4ded-95a5-851fd9f08852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:35.963294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:35.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:35.965291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:35.965291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:35.965291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:35.965291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 381/769] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:35.965867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717996ac-f783-426d-8759-47dbcc7245b6 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:35.966035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717996ac-f783-426d-8759-47dbcc7245b6 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:35.968755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717996ac-f783-426d-8759-47dbcc7245b6 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:35.968755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 382/770] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:35.974126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717996ac-f783-426d-8759-47dbcc7245b6 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:35.991837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd408c-fa26-4a6b-99a9-1ff71e8d8495 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.033797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fef70a-a851-4b8c-a6ba-b973924a5217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.034427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fef70a-a851-4b8c-a6ba-b973924a5217 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.034701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fef70a-a851-4b8c-a6ba-b973924a5217 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.034997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fef70a-a851-4b8c-a6ba-b973924a5217 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.035698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fef70a-a851-4b8c-a6ba-b973924a5217 None admin] RBAC: Adding query filter params (interface=public, service_id=f9c6471fd40a4767b19fcb2446e4524a, region_id=RegionOne) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:36.036005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fef70a-a851-4b8c-a6ba-b973924a5217 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.037399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fef70a-a851-4b8c-a6ba-b973924a5217 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.042534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:36.049258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 389/771] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:46:35 2022] GET /identity/v3/endpoints?service_id=f9c6471fd40a4767b19fcb2446e4524a&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:36.049258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61c6ca-5c86-4840-826c-930b3537b7cd None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.049258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61c6ca-5c86-4840-826c-930b3537b7cd None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.049258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61c6ca-5c86-4840-826c-930b3537b7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.067474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7e61c6ca-5c86-4840-826c-930b3537b7cd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:36.068746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7e61c6ca-5c86-4840-826c-930b3537b7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:36.089618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717996ac-f783-426d-8759-47dbcc7245b6 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.125044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.125646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.125915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.126189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.127376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:36.127659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.129051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.144228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 390/772] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:36.144228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61c6ca-5c86-4840-826c-930b3537b7cd None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.144800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 383/773] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:36.152944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.217837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:36.249612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61c6ca-5c86-4840-826c-930b3537b7cd None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.284657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ef696-5567-4001-ae7a-62d8291dfdab None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.285555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.286299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.286673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.287066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.287676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.289139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:36.298236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:36.299865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:36.300820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 391/774] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:36.301618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.331144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:36.354850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.355833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.356311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.357193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.358048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.360332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.368247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 384/775] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/services => generated 772 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:36.369820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.433572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896a9195-74a1-4f4c-a97b-4ba688877ac0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.490167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.490800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.491064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.491259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.494509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:36.499670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.501197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:36.515830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:36.516877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:36.517852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:36.518272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 392/776] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:36.519025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.536358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9b3e3-8704-4760-a250-952d7ed06b47 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.575940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.576579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.576878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.577151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.577796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] RBAC: Adding query filter params (name=network) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:36.578108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.579524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.589919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 385/777] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:36.590908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.640625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9a7f5-8ea0-4bb8-b15f-ed3faf8c0323 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.654193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:36.686261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.686763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.687049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.687323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.698990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:36.705116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:36.706205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:36.707042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:36.710858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 393/778] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:36.711773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.713658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a51d34-6311-400a-a8f2-69a3bb01ce23 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.750310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.751406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.751949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.752392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.753365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:36.753842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.755513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.769778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 386/779] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:36.775514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.785173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:36.828026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a7903-ae26-4d46-a385-20335c94f361 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.865979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.866594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.866933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.867205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.868025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:36.868333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.869840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.880502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 394/780] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:36.881535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.884758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5b6c0-639b-43e5-9308-19c12262cf1b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:36.900153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:36.926437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:36.927420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:36.927892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:36.928352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:36.929454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:36.930107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:36.931678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:36.944966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 387/781] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:36.946441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:36.997954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da042fa-a01e-458d-a96e-6af851e5e1c2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:37.020498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:37.036443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:37.037080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.037353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.037616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.038364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:37.038641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:37.040072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:37.052748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 395/782] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/role_assignments?user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:37.052748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:37.077911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55171b0f-cd7c-4093-a83e-5e8c1c291978 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:37.146249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:37.147255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.147727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.148188 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.149139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] RBAC: Adding query filter params (interface=public, service_id=fd146c4cd38b4e5d8ccd11197cb88ef4, region_id=RegionOne) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:37.149643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:37.151224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:37.165204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d48f7-fb8c-4145-bb7d-0e997588f9e6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:37.169123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 388/783] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:46:36 2022] GET /identity/v3/endpoints?service_id=fd146c4cd38b4e5d8ccd11197cb88ef4&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:37.174882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:37.194289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:37.194931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.195205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.195471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.207415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:37.209113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:37.225215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:37.226227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:37.227092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:37.227092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:37.227092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:37.227092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 396/784] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:37.233389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:37.250505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:37.303231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:37.383460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75b9e-eca2-4151-805f-c518be821d40 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:37.416587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe4223-1963-4a4f-9cdd-36582c739f10 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:37.457440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905140d-f9f8-43bc-a9f1-b9a5e66a6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:37.458059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905140d-f9f8-43bc-a9f1-b9a5e66a6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.458334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905140d-f9f8-43bc-a9f1-b9a5e66a6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.458602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905140d-f9f8-43bc-a9f1-b9a5e66a6dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.459252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905140d-f9f8-43bc-a9f1-b9a5e66a6dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:37.459552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905140d-f9f8-43bc-a9f1-b9a5e66a6dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:37.460960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905140d-f9f8-43bc-a9f1-b9a5e66a6dc1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:37.476387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 397/785] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:37.480908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1335429-5dd3-4600-b880-bb185d363b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:37.481762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727941ee-ce05-4aaf-865e-44052a90d024 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.482028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1335429-5dd3-4600-b880-bb185d363b08 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.482132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727941ee-ce05-4aaf-865e-44052a90d024 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.482311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727941ee-ce05-4aaf-865e-44052a90d024 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.482765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1335429-5dd3-4600-b880-bb185d363b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.483238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1335429-5dd3-4600-b880-bb185d363b08 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.487727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 398/786] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:37.487727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727941ee-ce05-4aaf-865e-44052a90d024 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:37.488330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1335429-5dd3-4600-b880-bb185d363b08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:37.490050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1335429-5dd3-4600-b880-bb185d363b08 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:37.496180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 389/787] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:46:37 2022] GET /identity/v3/services => generated 772 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:37.497803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71979f-8d32-4e70-8400-5b5d09e3bb2a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.498260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71979f-8d32-4e70-8400-5b5d09e3bb2a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.498717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71979f-8d32-4e70-8400-5b5d09e3bb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.524767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-7c71979f-8d32-4e70-8400-5b5d09e3bb2a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:37.527253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7c71979f-8d32-4e70-8400-5b5d09e3bb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:37.658907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 390/788] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:37.677128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:37.689052 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727941ee-ce05-4aaf-865e-44052a90d024 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:37.730656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:37.731291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.731572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.731847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.739991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:37.741482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:37.754857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:37.755805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:37.756749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 399/789] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:37.761786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71979f-8d32-4e70-8400-5b5d09e3bb2a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:37.797784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:37.877626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71979f-8d32-4e70-8400-5b5d09e3bb2a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:37.947372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:37.947910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:37.948186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:37.948452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:37.949224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:37.949529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:37.950973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:37.964161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:37.992330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 391/790] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 15:46:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:38.000569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:38.139601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40f50-77ff-4f45-82ea-e2754bcc7bf2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:38.165073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:38.179385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b0cac-4eae-4ad4-ba1d-740a67b22b69 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:38.201571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:38.202123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:38.202583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:38.202866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:38.225317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:38.233529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:38.234551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:38.235391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:38.236683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 400/791] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:38.238144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:38.238867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:38.239268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:38.239602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:38.241402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:38.260328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:38.261835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:38.285078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:38.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:38.287181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:38.288128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:38.288128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:38.288128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:38.288128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 392/792] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:38.293642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:38.443429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9fce2-6b23-48dc-9e36-996601bf66a3 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:38.507295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f663ed0-4957-4843-958f-46790e10d8ca None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:38.523369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:38.524304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:38.524801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:38.525265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:38.526149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:38.526649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:38.528225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:38.551332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 401/793] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:38.553209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:38.586699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:38.587665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:38.589089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:38.589557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:38.590811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:38.593924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:38.595645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:38.624096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 393/794] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:38.629618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:38.635208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:38.670198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:38.696557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59807c09-aa2a-4517-9272-cf388f68fd92 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:38.739814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:38.740866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:38.741348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:38.741838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:38.767876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:38.769725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:38.794302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:38.795322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:38.796201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:38.796201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:38.796201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:38.796201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 402/795] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:38.802577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:38.815388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2238d0-013b-4df6-9acb-631b7ac69a74 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:38.854267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441dc06-b960-4a31-90fa-49b21528119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:38.855295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441dc06-b960-4a31-90fa-49b21528119e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:38.856384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441dc06-b960-4a31-90fa-49b21528119e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:38.856918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441dc06-b960-4a31-90fa-49b21528119e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:38.865264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441dc06-b960-4a31-90fa-49b21528119e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:38.867598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441dc06-b960-4a31-90fa-49b21528119e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-4441dc06-b960-4a31-90fa-49b21528119e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:38.881228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:38.882144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:38.883027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 394/796] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:38.883973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35b520-a266-49cc-9fa5-45d630703f22 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:38.884450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35b520-a266-49cc-9fa5-45d630703f22 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:38.884553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35b520-a266-49cc-9fa5-45d630703f22 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:38.885873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 395/797] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:38.889535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35b520-a266-49cc-9fa5-45d630703f22 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:38.966277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da8916-2b86-45a4-b45a-b4e8889834b4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:38.976483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:39.001996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:39.008215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46efbf-cd96-47cd-809c-fac6f378fbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:39.009007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46efbf-cd96-47cd-809c-fac6f378fbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.009647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46efbf-cd96-47cd-809c-fac6f378fbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.010173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46efbf-cd96-47cd-809c-fac6f378fbd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.011114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46efbf-cd96-47cd-809c-fac6f378fbd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:39.011797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46efbf-cd96-47cd-809c-fac6f378fbd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:39.013247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46efbf-cd96-47cd-809c-fac6f378fbd5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:39.031008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 403/798] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:39.032549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c130a2-4152-48b5-8b71-9fdc46468817 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.033011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c130a2-4152-48b5-8b71-9fdc46468817 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.033291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35b520-a266-49cc-9fa5-45d630703f22 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:39.033624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c130a2-4152-48b5-8b71-9fdc46468817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.058343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-59c130a2-4152-48b5-8b71-9fdc46468817 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:39.059520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-59c130a2-4152-48b5-8b71-9fdc46468817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:39.079293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:39.080647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.081122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.081562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.082528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:39.083515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:39.085513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:39.103690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 396/799] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:39.104813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:39.143258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 404/800] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:39.144183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c130a2-4152-48b5-8b71-9fdc46468817 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:39.224677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01273aa3-f2ec-4e0f-b81f-a789d10ff23a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:39.249383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:39.263842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:39.264810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.265304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.265765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.274743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:39.276501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:39.288636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:39.289594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:46:39.290765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:39.301249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 397/801] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:46:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:39.310012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:39.322247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c130a2-4152-48b5-8b71-9fdc46468817 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:39.343325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:39.369397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:39.370067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.370342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.370609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/7701e2653e4b4ca7a842887c0526862f/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.375597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=7701e2653e4b4ca7a842887c0526862f, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:39.378015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:39.404631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 405/802] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:39 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/7701e2653e4b4ca7a842887c0526862f/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:39.473665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a4bc3-2879-4cf3-88cb-e78a401a5449 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:39.516714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:39.517442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.517987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.518458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.519273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:39.519631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:39.521184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:39.552688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 398/803] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:46:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:39.558487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:39.641228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:39.683161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c0ec1-00fb-47c5-9a77-37bc06b5eae4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:39.691019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:39.725722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cec364-36b8-41eb-86ab-13b48b91aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:39.725722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cec364-36b8-41eb-86ab-13b48b91aed0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.725722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cec364-36b8-41eb-86ab-13b48b91aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.726907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cec364-36b8-41eb-86ab-13b48b91aed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.763588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b00646-cda0-42da-9173-0ab66b446a06 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:39.789282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:39.802944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d477897-b262-4d08-b9d7-c091acfaf8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:39.803853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d477897-b262-4d08-b9d7-c091acfaf8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.804271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d477897-b262-4d08-b9d7-c091acfaf8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.804776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d477897-b262-4d08-b9d7-c091acfaf8a1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.805505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d477897-b262-4d08-b9d7-c091acfaf8a1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:39.807024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d477897-b262-4d08-b9d7-c091acfaf8a1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:39.813117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cec364-36b8-41eb-86ab-13b48b91aed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:39.815243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cec364-36b8-41eb-86ab-13b48b91aed0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1d477897-b262-4d08-b9d7-c091acfaf8a1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:39.816893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:39.818052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 15:46:39.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:39.821527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 399/804] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Oct 3 15:46:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:39.822865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791d335-e891-4386-87fa-b2cfd78d7c06 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.823269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791d335-e891-4386-87fa-b2cfd78d7c06 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.823662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791d335-e891-4386-87fa-b2cfd78d7c06 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.825295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 400/805] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:39.826769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791d335-e891-4386-87fa-b2cfd78d7c06 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:39.945565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 406/806] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:46:39.947279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b51c-b3d0-4f7e-b9f7-9f793227c32c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:39.947690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b51c-b3d0-4f7e-b9f7-9f793227c32c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:39.948087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b51c-b3d0-4f7e-b9f7-9f793227c32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:39.950943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791d335-e891-4386-87fa-b2cfd78d7c06 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:39.953873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:39.971918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-9980b51c-b3d0-4f7e-b9f7-9f793227c32c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:39.974122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9980b51c-b3d0-4f7e-b9f7-9f793227c32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:40.002153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:40.003235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.003621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.003993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.004763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:40.005183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:40.006773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:40.016908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 401/807] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 15:46:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:40.018028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.060356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 407/808] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:40.061471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b51c-b3d0-4f7e-b9f7-9f793227c32c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.077794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:40.220853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0d643-5d73-4fc0-bb22-960397a7fa71 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:40.232252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b51c-b3d0-4f7e-b9f7-9f793227c32c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:40.264332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa70c5f-7d36-473c-9b43-fd9ba265efa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:40.265363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa70c5f-7d36-473c-9b43-fd9ba265efa7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.265649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa70c5f-7d36-473c-9b43-fd9ba265efa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.265919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa70c5f-7d36-473c-9b43-fd9ba265efa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.274915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801f7f8-af4c-4e4f-a676-f68db8a253d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:40.275766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f7f8-af4c-4e4f-a676-f68db8a253d0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.276232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f7f8-af4c-4e4f-a676-f68db8a253d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.276643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801f7f8-af4c-4e4f-a676-f68db8a253d0 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.277918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f7f8-af4c-4e4f-a676-f68db8a253d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:40.278062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa70c5f-7d36-473c-9b43-fd9ba265efa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:40.278499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f7f8-af4c-4e4f-a676-f68db8a253d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:40.279457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa70c5f-7d36-473c-9b43-fd9ba265efa7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:40.280348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801f7f8-af4c-4e4f-a676-f68db8a253d0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:40.287070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 408/809] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Oct 3 15:46:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:40.289125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840cf04-701e-4bd7-a505-29a1376b5ca6 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.290022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840cf04-701e-4bd7-a505-29a1376b5ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.290472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840cf04-701e-4bd7-a505-29a1376b5ca6 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.292359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 409/810] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:40.294496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840cf04-701e-4bd7-a505-29a1376b5ca6 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-4fa70c5f-7d36-473c-9b43-fd9ba265efa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:40.307058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:40.308016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:40.309011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:40.309011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:40.309011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:40.309011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 402/811] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:40.309827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277b6d-97d8-4039-b536-a5ad8154e3a2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.310107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277b6d-97d8-4039-b536-a5ad8154e3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.310429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277b6d-97d8-4039-b536-a5ad8154e3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.340568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-7d277b6d-97d8-4039-b536-a5ad8154e3a2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:40.342087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7d277b6d-97d8-4039-b536-a5ad8154e3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:40.364688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:40.394658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:40.428327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840cf04-701e-4bd7-a505-29a1376b5ca6 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:40.440799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 403/812] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:40.440799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277b6d-97d8-4039-b536-a5ad8154e3a2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.489205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:40.489809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.490081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.490348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.491001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:40.492347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:40.544151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 410/813] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:46:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:40.583682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277b6d-97d8-4039-b536-a5ad8154e3a2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:40.627873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:40.628938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.629425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.629946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.630831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:40.631332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:40.632978 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:40.647189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 404/814] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:40.654999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.660848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.671716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:40.774390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0798f-2372-4193-94c6-41d893a099f8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:40.823559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:40.824622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.825148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.825617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.847855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:40.858325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:40.859276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:40.860089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:40.860740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 405/815] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:40.862198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.870775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca14f44-d072-45b4-9fd0-ea305a3bcd96 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:40.927865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:40.942111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:40.947081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:40.948381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:40.948381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:40.948381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:40.957109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:40.958670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:40.972515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:40.973682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:40.974556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 411/816] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:40.975483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:40.985562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2108e-f1c6-4eda-a1d0-96867e28d1a0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.021814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.022826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.023378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.023842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] PATH_INFO: `/v3/services/network` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.024684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.026343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:41.036291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:41.037359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 15:46:41.038305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:41.040464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 406/817] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:40 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:41.041440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.108155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552ac33-541f-4f5b-8ae4-0c5ed647b746 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.147782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.148393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.148667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.149335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.150159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:41.150606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.151976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.152542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661fd94-0099-4c51-a335-73bc6e23aa12 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.165956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 412/818] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:41.166835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.189511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:41.198485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.199354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.199632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.199899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.200594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:41.201175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.203309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.219019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 407/819] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:41.220807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.232146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:41.304224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e1ee9-9911-4421-96c9-e1a84ce8fcaf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.343375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.344118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.344489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.344920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.346310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:41.346681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.348111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.353563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 413/820] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:41.354483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.362453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574f99b-b64e-4642-b0bf-ce5b18165496 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.400001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.401130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.401589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.402032 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.414006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.416009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:41.432522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:41.433534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:41.434413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:41.434413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:41.434413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:41.436342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 408/821] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:41.437468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.474261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c15c89-ed19-4c44-ad33-b2b1523cc7f8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.488651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:41.495910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:41.517954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.518779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.519197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.519654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.520618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=ed959e4bdaf44132b05ccaec68c41172) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:41.521096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.522756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.530261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 414/822] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:41 2022] GET /identity/v3/role_assignments?user.id=ed959e4bdaf44132b05ccaec68c41172&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:41.531403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.546591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61da53-b428-4417-8111-7ef01ddf1213 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.590297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.590954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.591223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.591441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.592094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:41.592396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.594681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.611801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 409/823] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:41.617910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.708444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d1afa-c142-4f2d-95db-e4f57242b557 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.779459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.780463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.780986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.781472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.782327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:41.783375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.787327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.809497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 415/824] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:41.810981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b2c11-c956-4709-ac88-098b4302b1e7 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:41.811854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:41.822679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:41.861339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:41.862198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:41.862479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:41.862750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:41.863430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:41.864837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:41.875105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:41.907542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 410/825] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:46:41 2022] POST /identity/v3/endpoints => generated 337 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:46:41.959765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6bbac-58fc-41f3-9696-7f681ca13152 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:42.008951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:42.010841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:42.010841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:42.010841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:42.028757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:42.030557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:42.051577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:42.053722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:42.054657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 416/826] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:46:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:42.060415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:42.195211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb30ed-979a-4c0f-bbae-3549001da467 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:42.238429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9dfe2-af7e-47a7-9795-f051c875a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:42.239205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9dfe2-af7e-47a7-9795-f051c875a162 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:42.239579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9dfe2-af7e-47a7-9795-f051c875a162 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:42.239953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9dfe2-af7e-47a7-9795-f051c875a162 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:42.240829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9dfe2-af7e-47a7-9795-f051c875a162 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:42.241238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9dfe2-af7e-47a7-9795-f051c875a162 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:42.242784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9dfe2-af7e-47a7-9795-f051c875a162 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:42.262495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 411/827] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 15:46:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:42.269194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:42.387625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:42.394325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf88ad-0dd8-41d5-95c1-57d2205c3e4f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:42.411028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b69ea4-abc7-4645-b1fa-f24d40949a12 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:42.412155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b69ea4-abc7-4645-b1fa-f24d40949a12 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:42.412155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b69ea4-abc7-4645-b1fa-f24d40949a12 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:42.413800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 412/828] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:42.419926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee322b-5122-4019-8709-a988e07c5f28 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:42.420209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee322b-5122-4019-8709-a988e07c5f28 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:42.420484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee322b-5122-4019-8709-a988e07c5f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:42.443574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:42.444543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:42.445041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:42.445561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:42.445804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-21ee322b-5122-4019-8709-a988e07c5f28 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:42.448145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-21ee322b-5122-4019-8709-a988e07c5f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:42.459191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:42.460958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:42.465937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:42.478104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:42.479361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:42.480264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:42.480264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:42.480264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:42.484974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 417/829] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:42.486337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:42.541990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 413/830] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:42.602618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1ca2f-ad4b-4f8b-871b-28c042701299 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:42.639635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:42.640239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:42.640508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:42.640798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:42.641442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:42.641738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:42.643982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:42.660828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 418/831] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:42.662441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee322b-5122-4019-8709-a988e07c5f28 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:42.773760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee322b-5122-4019-8709-a988e07c5f28 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:42.811023 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:42.812118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:42.812623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:42.813165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:42.820907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:42.822665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:42.835101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:42.835995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:42.836922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:42.839269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 414/832] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:42.842417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:42.846220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:42.856289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:42.862681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:42.966393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1aa229-1a39-4b5a-adbf-3918db16cf79 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:42.985636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd73b1-30ca-4f68-8298-3ebd04967891 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:43.011208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.011881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.012164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.012433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.013153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:43.013452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:43.015211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:43.031988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.032167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 415/833] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:43.033113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.033600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.033706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.034300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.049298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:43.057089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:43.058051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:46:43.058877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:43.064764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 419/834] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:43.068317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.115047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:43.151009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:43.156611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b96bc-96fe-4fc6-85fc-5d0b1068deea None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:43.192789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c929d14-d8bb-49e9-b614-e88a81c1fa77 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:43.202002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b6f05-981e-42f3-a643-b8b699f7a444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.202686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b6f05-981e-42f3-a643-b8b699f7a444 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.203051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b6f05-981e-42f3-a643-b8b699f7a444 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.203784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b6f05-981e-42f3-a643-b8b699f7a444 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/cd0c6adf0bb34ba0b33bad9d46b47ac8/roles/e3ae1cd82e064f38a35cc76b525ffbbf` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.220041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b6f05-981e-42f3-a643-b8b699f7a444 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=cd0c6adf0bb34ba0b33bad9d46b47ac8, role_id=e3ae1cd82e064f38a35cc76b525ffbbf)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:43.222480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b6f05-981e-42f3-a643-b8b699f7a444 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:43.240382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.241025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.241299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.241730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.242231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 416/835] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:43 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/cd0c6adf0bb34ba0b33bad9d46b47ac8/roles/e3ae1cd82e064f38a35cc76b525ffbbf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:43.242403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:43.242669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:43.244116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:43.246201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af2c2-14b1-47b6-85b5-7c3f70b4153c None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.246855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af2c2-14b1-47b6-85b5-7c3f70b4153c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.247331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808af2c2-14b1-47b6-85b5-7c3f70b4153c None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.250630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 417/836] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:43.260858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5177d-d517-4093-a4ef-7c594836c8e1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.260858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5177d-d517-4093-a4ef-7c594836c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.260858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5177d-d517-4093-a4ef-7c594836c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.261199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 420/837] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:43.261199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.283840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cbf5177d-d517-4093-a4ef-7c594836c8e1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:43.285879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cbf5177d-d517-4093-a4ef-7c594836c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:43.397160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c06a14-32d7-4b93-9295-4a4e89c59200 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:43.400843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 418/838] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:43.437032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:43.444250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.444948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.445223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.445488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.457375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:43.458818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:43.473690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:43.474653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:43.475526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:43.475526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:43.475526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:43.475526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 421/839] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:43.480997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5177d-d517-4093-a4ef-7c594836c8e1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.537039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:43.596469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5177d-d517-4093-a4ef-7c594836c8e1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:43.640374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.641026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.641306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.641576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.642828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:43.643355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:43.644981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:43.657498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 419/840] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:43.663716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.684176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.801986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799872-317a-4de4-8e4e-0944f6e46736 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:43.812528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62ffb0-3432-4b52-85b7-f61c4c51b78d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:43.824892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:43.839487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.840115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.840389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.840655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.850165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:43.851600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:43.853835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99661aca-c112-447f-8372-003a680c415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:43.854712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661aca-c112-447f-8372-003a680c415c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:43.855149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661aca-c112-447f-8372-003a680c415c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:43.855565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99661aca-c112-447f-8372-003a680c415c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:43.856296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99661aca-c112-447f-8372-003a680c415c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:43.857846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99661aca-c112-447f-8372-003a680c415c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:43.863739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:43.864943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:43.866199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 422/841] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:46:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-99661aca-c112-447f-8372-003a680c415c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:43.871281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:43.872301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:43.873210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 420/842] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:43.873992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.877930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99661aca-c112-447f-8372-003a680c415c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:43.927514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:44.004181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147c9b-b76a-4bd5-839c-c32dd7b9688e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:44.030426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99661aca-c112-447f-8372-003a680c415c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:44.046315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:44.047227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.047729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.048002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.048828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:44.049135 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:44.050558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:44.068941 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 423/843] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Oct 3 15:46:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:44.076765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:44.076925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:44.077009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.077009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.077009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.077009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:44.077009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:44.077582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:44.092807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 421/844] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:44.094231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:44.143732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:44.195475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:44.199532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f72c4d-75ac-4942-a69b-6f5fd21ba116 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:44.211173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250c4e2-a4b3-4444-a979-5e5542a24002 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:44.240254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:44.241548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.242214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.242629 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.248395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76eed6-795f-4fa5-9112-cb3be6a12e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:44.249019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76eed6-795f-4fa5-9112-cb3be6a12e32 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.249294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76eed6-795f-4fa5-9112-cb3be6a12e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.249561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76eed6-795f-4fa5-9112-cb3be6a12e32 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.250183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76eed6-795f-4fa5-9112-cb3be6a12e32 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:44.250477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76eed6-795f-4fa5-9112-cb3be6a12e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:44.251892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76eed6-795f-4fa5-9112-cb3be6a12e32 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:44.256531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 422/845] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:44.257671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902008-f765-4c07-86a3-cdaea5bb7685 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.257948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902008-f765-4c07-86a3-cdaea5bb7685 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.258219 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902008-f765-4c07-86a3-cdaea5bb7685 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.259426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 423/846] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:44.265361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654df7ce-0098-4e13-83e5-1dbb29d416aa None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.265698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654df7ce-0098-4e13-83e5-1dbb29d416aa None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.265917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654df7ce-0098-4e13-83e5-1dbb29d416aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.290892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-654df7ce-0098-4e13-83e5-1dbb29d416aa None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:44.292304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-654df7ce-0098-4e13-83e5-1dbb29d416aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:44.341056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:44.343544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:44.391214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 424/847] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:44.461019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:44.464813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 424/848] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 15:46:44.470778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654df7ce-0098-4e13-83e5-1dbb29d416aa None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:44.505983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:44.584446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654df7ce-0098-4e13-83e5-1dbb29d416aa None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:44.621214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:44.621843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.622159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.622429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.633525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:44.634976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:44.652227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:44.653225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:44.653225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:44.653225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:44.653225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 425/849] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:44.658984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:44.677739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:44.773924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211e4d5-dd98-4739-8abc-b588e50ee305 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:44.792268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:44.815912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cc58-44f0-4039-bcbb-1eab272ce7de None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:44.820331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:44.820973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.821379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.821659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.822306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:44.822602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:44.823973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:44.843538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 425/850] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:44.844534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:44.865512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:44.866496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:44.866996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:44.867481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:44.868847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:44.871003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:44.884752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:44.885783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:44.886683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:44.894815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 426/851] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Oct 3 15:46:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:44.896515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:44.916341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:44.963167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a6049-c6c5-4aa3-9c89-f73fb74a987c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:45.003727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:45.004456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:45.005046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:45.005131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:45.015361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:45.017496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:45.031766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7694ccc-454d-4ed5-a357-c14a7a3ff6f1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:45.035798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:45.036671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:45.037667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 426/852] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:45.043898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:45.078508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:45.080048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:45.080816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:45.081298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:45.082168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:45.082713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:45.084364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:45.089892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 427/853] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:45.095831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:45.114325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:45.189074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38710140-c008-49a1-8f0b-d6b220553298 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:45.209333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5412b5e-0ef8-4191-9ac6-9453742be495 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:45.218490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:45.240372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:45.241448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:45.241735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:45.242006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:45.242977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:45.243289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:45.245020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:45.250253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d696a-b19b-4e63-bcc2-925f13ae414a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:45.251097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d696a-b19b-4e63-bcc2-925f13ae414a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:45.251650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d696a-b19b-4e63-bcc2-925f13ae414a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:45.252180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d696a-b19b-4e63-bcc2-925f13ae414a None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:45.253041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d696a-b19b-4e63-bcc2-925f13ae414a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:45.253539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d696a-b19b-4e63-bcc2-925f13ae414a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:45.255122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d696a-b19b-4e63-bcc2-925f13ae414a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:45.260763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 427/854] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:45.269742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 428/855] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:45.269938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:45.389987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f461c7-c2ac-4980-a5ac-f633552e95f1 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:45.433443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959ff05-26fa-4439-930e-ed763dcf3102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:45.434553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ff05-26fa-4439-930e-ed763dcf3102 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:45.435053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ff05-26fa-4439-930e-ed763dcf3102 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:45.435521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959ff05-26fa-4439-930e-ed763dcf3102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:45.436494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959ff05-26fa-4439-930e-ed763dcf3102 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=7701e2653e4b4ca7a842887c0526862f) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:45.437057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959ff05-26fa-4439-930e-ed763dcf3102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:45.438779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a959ff05-26fa-4439-930e-ed763dcf3102 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:45.446599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 428/856] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:45 2022] GET /identity/v3/role_assignments?user.id=7701e2653e4b4ca7a842887c0526862f&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:45.516919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93088-f760-4d2e-8f3d-c00d63e96231 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:45.517859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93088-f760-4d2e-8f3d-c00d63e96231 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:45.518202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef93088-f760-4d2e-8f3d-c00d63e96231 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:45.519550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 429/857] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:45.525639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379e54-2f09-4e8b-ae18-6b73c4d5bb01 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:45.525951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379e54-2f09-4e8b-ae18-6b73c4d5bb01 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:45.526199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c379e54-2f09-4e8b-ae18-6b73c4d5bb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:45.546192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7c379e54-2f09-4e8b-ae18-6b73c4d5bb01 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:45.547408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7c379e54-2f09-4e8b-ae18-6b73c4d5bb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:45.596789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:45.627737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 429/858] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:45.909556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef93088-f760-4d2e-8f3d-c00d63e96231 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:45.995242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:46.021351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef93088-f760-4d2e-8f3d-c00d63e96231 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:46.057245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:46.057877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:46.058151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:46.058417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:46.069543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:46.074913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:46.077027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:46.079062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 430/859] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:46.084757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c379e54-2f09-4e8b-ae18-6b73c4d5bb01 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:46.187041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c379e54-2f09-4e8b-ae18-6b73c4d5bb01 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:46.222888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:46.224192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:46.224547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:46.224844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:46.225614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:46.226136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:46.228107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:46.240549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 430/860] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:46:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:46.246596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:46.363415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b1ab2-624e-4282-95be-bcd7d8fe454f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:46.408390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3628b04-0fd5-42d9-9aec-7d12458ad393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:46.409234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3628b04-0fd5-42d9-9aec-7d12458ad393 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:46.409436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3628b04-0fd5-42d9-9aec-7d12458ad393 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:46.409681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3628b04-0fd5-42d9-9aec-7d12458ad393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:46.413473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:46.428749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3628b04-0fd5-42d9-9aec-7d12458ad393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:46.428749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3628b04-0fd5-42d9-9aec-7d12458ad393 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-b3628b04-0fd5-42d9-9aec-7d12458ad393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:46.442020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:46.444135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:46.446441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:46.446441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:46.446441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:46.446441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 431/861] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:46.450211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:46.497147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:46.526929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8b8fc-6e55-4612-88b2-c92b0b56307f None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:46.527210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8b8fc-6e55-4612-88b2-c92b0b56307f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:46.527483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8b8fc-6e55-4612-88b2-c92b0b56307f None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:46.536753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 432/862] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:46.536753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da1963-5ef5-4721-befe-6d62713d7ba1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:46.536753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da1963-5ef5-4721-befe-6d62713d7ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:46.536753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da1963-5ef5-4721-befe-6d62713d7ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:46.561898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92851a44-292e-408a-8b9c-be72e835872a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:46.574080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-71da1963-5ef5-4721-befe-6d62713d7ba1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:46.575280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-71da1963-5ef5-4721-befe-6d62713d7ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:46.600748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:46.601385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:46.601663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:46.601934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:46.602581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:46.602889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:46.604529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:46.619306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 431/863] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:46.624933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:46.732735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:46.734568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 433/864] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:46.740969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8f451-b464-4b37-a2be-c10b750e2ca7 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:46.777859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:46.778880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:46.779365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:46.779842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:46.787660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:46.789392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:46.807300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:46.808302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:46.809297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 432/865] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:46:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:46.816311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da1963-5ef5-4721-befe-6d62713d7ba1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:46.984115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da1963-5ef5-4721-befe-6d62713d7ba1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:47.003220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:47.021850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:47.022629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.022999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.023367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.024207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:47.024614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:47.027344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:47.045111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 434/866] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 15:46:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:47.050778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:47.084118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:47.170338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c783c6-c3ca-49d0-ad3a-978b0d5b0a17 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:47.205191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5259f-6b72-43b6-8ac8-eeac16d703ea None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:47.216808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:47.217883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.218405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.218847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.243846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:47.244479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.244867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.245210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.245953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:47.247526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:47.250718 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:47.265730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 435/867] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:46:47 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:46:47.301051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:47.302574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:47.413227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 433/868] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:46:47.418852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:47.461138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:47.540172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b8dfd-0094-4c26-ba5d-48dc615050f0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:47.581157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f69aa-c02d-46f3-a98b-8c3ef6797809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:47.581812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f69aa-c02d-46f3-a98b-8c3ef6797809 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.582094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f69aa-c02d-46f3-a98b-8c3ef6797809 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.582361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f69aa-c02d-46f3-a98b-8c3ef6797809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.593713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f69aa-c02d-46f3-a98b-8c3ef6797809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:47.595244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f69aa-c02d-46f3-a98b-8c3ef6797809 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-a05f69aa-c02d-46f3-a98b-8c3ef6797809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:47.611731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:47.613901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:47.616019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:47.616019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:47.616019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:47.616019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 436/869] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:47.622017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:47.687222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:47.693252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e45c2-9c3f-455d-a348-dd6dc39e271f None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.693522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e45c2-9c3f-455d-a348-dd6dc39e271f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.693796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e45c2-9c3f-455d-a348-dd6dc39e271f None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.695292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 437/870] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:47.700803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5a3d3-0868-4c03-9509-5a95a56abc49 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.701088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5a3d3-0868-4c03-9509-5a95a56abc49 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.701364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5a3d3-0868-4c03-9509-5a95a56abc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.721959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d3f5a3d3-0868-4c03-9509-5a95a56abc49 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:47.723414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d3f5a3d3-0868-4c03-9509-5a95a56abc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:47.736083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e4bb0-6509-4003-ac99-206a02598373 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:47.783377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:47.784004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.784280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.784766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.785802 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:47.786119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:47.787557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:47.802820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 434/871] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:47.808449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:47.810847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 438/872] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:47.895296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248be2-b57b-41e4-a935-774f0dfea702 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.895296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248be2-b57b-41e4-a935-774f0dfea702 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.897676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248be2-b57b-41e4-a935-774f0dfea702 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.897676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 439/873] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:47.908075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c57b38-3663-4601-92b4-1cab06869443 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.908075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c57b38-3663-4601-92b4-1cab06869443 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.908075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c57b38-3663-4601-92b4-1cab06869443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.923169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72843634-623b-4d33-abb8-c9c84498ad1b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:47.925175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-77c57b38-3663-4601-92b4-1cab06869443 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:47.926576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-77c57b38-3663-4601-92b4-1cab06869443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:47.936689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:47.940973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:47.962127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:47.963153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:47.963428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:47.963694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:47.972553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:47.974290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:47.991446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:47.992391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:47.993306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:47.996806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 435/874] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:47.999658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:48.021205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 440/875] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:47 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:48.104767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb38f4-23ad-4ab2-a615-9290437d8754 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:48.139739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:48.140824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:48.141326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:48.142460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:48.143721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:48.144247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:48.145627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c57b38-3663-4601-92b4-1cab06869443 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:48.145907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:48.164513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 436/876] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:48.170243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:48.215229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:48.294569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae7f46-937c-4af9-b419-34a988c63d00 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:48.297027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c57b38-3663-4601-92b4-1cab06869443 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:48.332229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:48.332863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:48.333139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:48.333396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:48.334117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:48.334482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:48.335831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:48.338368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:48.338965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:48.339226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:48.339493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:48.340142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:48.341615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:48.353129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 437/877] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:48 2022] GET /identity/v3/role_assignments?user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:48.353129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:48.370825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 441/878] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:46:48 2022] POST /identity/v3/services => generated 269 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:48.481928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:48.485018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7d63-c516-4cd5-8ea8-21a2516ea735 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:48.532172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:48.534028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:48.534519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:48.534980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:48.535769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:48.537463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:48.547526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:48.548911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:48.549754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:48.551929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 438/879] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:48.558223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:48.564273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:48.676064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97265-08aa-4ea4-a764-34472035ef4f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:48.712778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ffed2-cd7a-4ea0-9dcb-ce7b7164b0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:48.713391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ffed2-cd7a-4ea0-9dcb-ce7b7164b0ac None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:48.713664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ffed2-cd7a-4ea0-9dcb-ce7b7164b0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:48.713929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ffed2-cd7a-4ea0-9dcb-ce7b7164b0ac None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:48.714573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ffed2-cd7a-4ea0-9dcb-ce7b7164b0ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:48.714874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ffed2-cd7a-4ea0-9dcb-ce7b7164b0ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:48.716249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ffed2-cd7a-4ea0-9dcb-ce7b7164b0ac None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:48.722436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 442/880] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:46:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:48.728829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:48.833576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e77e16-e003-4833-84b4-1165d9915c94 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:48.869576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8cf41-d029-44c8-9fb2-b963ec9e6403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:48.870522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8cf41-d029-44c8-9fb2-b963ec9e6403 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:48.871024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8cf41-d029-44c8-9fb2-b963ec9e6403 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:48.871498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8cf41-d029-44c8-9fb2-b963ec9e6403 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:48.872395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8cf41-d029-44c8-9fb2-b963ec9e6403 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:48.872931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8cf41-d029-44c8-9fb2-b963ec9e6403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:48.874514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8cf41-d029-44c8-9fb2-b963ec9e6403 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:48.884931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 439/881] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:46:48 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:49.586187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ef166-b6f3-4f89-982b-9105558a28ab None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:49.586677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ef166-b6f3-4f89-982b-9105558a28ab None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:49.586677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ef166-b6f3-4f89-982b-9105558a28ab None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:49.588211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 443/882] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:49.593348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbe503-5a90-45a0-ac70-373a04f558c4 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:49.594193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbe503-5a90-45a0-ac70-373a04f558c4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:49.594676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbe503-5a90-45a0-ac70-373a04f558c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:49.614657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-33dbe503-5a90-45a0-ac70-373a04f558c4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:49.616208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-33dbe503-5a90-45a0-ac70-373a04f558c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:49.634604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:49.690274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 440/883] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:49.963870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ef166-b6f3-4f89-982b-9105558a28ab None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:50.035466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:50.073338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ef166-b6f3-4f89-982b-9105558a28ab None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:50.115441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:50.116086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.116359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.116626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.117242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:50.118626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:50.128545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:50.129585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:50.130417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 444/884] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:50.135606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbe503-5a90-45a0-ac70-373a04f558c4 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:50.245163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbe503-5a90-45a0-ac70-373a04f558c4 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:50.288240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:50.288886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.289164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.289432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.290077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:50.290378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:50.291743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:50.297182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 441/885] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:50.309842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:50.434337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3254567-0b0d-4583-88c7-e83e15b7de3d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:50.473485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:50.474166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.474456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.474718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.475338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:50.475642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:50.477056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:50.483152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:50.483529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 445/886] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:50.489804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:50.515605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:50.609647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742234d-f73e-4be5-80e4-f71721a5df78 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:50.651336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4e855-5016-4c5e-9b73-e0d5c5b411e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:50.652238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e855-5016-4c5e-9b73-e0d5c5b411e5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.652517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e855-5016-4c5e-9b73-e0d5c5b411e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.652867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4e855-5016-4c5e-9b73-e0d5c5b411e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.653581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e855-5016-4c5e-9b73-e0d5c5b411e5 None admin] RBAC: Adding query filter params (interface=public, service_id=5f2bdac74bdd4be781540382e544d5f3, region_id=RegionOne) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:50.653936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e855-5016-4c5e-9b73-e0d5c5b411e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:50.655311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4e855-5016-4c5e-9b73-e0d5c5b411e5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:50.662931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 442/887] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:46:50 2022] GET /identity/v3/endpoints?service_id=5f2bdac74bdd4be781540382e544d5f3&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:50.668699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:50.793654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f21160-7f45-4991-bb03-875eb3883501 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:50.810152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7f0e0-cedc-4fa1-a731-19af6ac57e8e None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.810774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7f0e0-cedc-4fa1-a731-19af6ac57e8e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.811255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7f0e0-cedc-4fa1-a731-19af6ac57e8e None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.813077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 443/888] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:50.819403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db2da8-6d9e-4e3a-bd97-c828e7392464 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.819976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db2da8-6d9e-4e3a-bd97-c828e7392464 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.820470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db2da8-6d9e-4e3a-bd97-c828e7392464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.831785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc80146-449b-4441-9a1f-a325b0199fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:50.832397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc80146-449b-4441-9a1f-a325b0199fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.832674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc80146-449b-4441-9a1f-a325b0199fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.833054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc80146-449b-4441-9a1f-a325b0199fe5 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.833676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc80146-449b-4441-9a1f-a325b0199fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:50.835065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc80146-449b-4441-9a1f-a325b0199fe5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:50.841882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-61db2da8-6d9e-4e3a-bd97-c828e7392464 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:50.842204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 446/889] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:46:50 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:46:50.843530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-61db2da8-6d9e-4e3a-bd97-c828e7392464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:50.852479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5604d-fabd-4ee3-93b0-7e877c68993d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.852856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5604d-fabd-4ee3-93b0-7e877c68993d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.853070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5604d-fabd-4ee3-93b0-7e877c68993d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.854784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 447/890] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:50.860758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05340833-e328-4434-b757-afd6c7fa2b35 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:50.860957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05340833-e328-4434-b757-afd6c7fa2b35 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:50.861194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05340833-e328-4434-b757-afd6c7fa2b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:50.882331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-05340833-e328-4434-b757-afd6c7fa2b35 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:50.883575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-05340833-e328-4434-b757-afd6c7fa2b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:50.937162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 444/891] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:50.953305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:50.966525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 448/892] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:51.200478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db2da8-6d9e-4e3a-bd97-c828e7392464 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.215822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:51.255192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05340833-e328-4434-b757-afd6c7fa2b35 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.310993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db2da8-6d9e-4e3a-bd97-c828e7392464 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:51.348395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26ede0-e0f2-41fd-bbac-a0d4694ea7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:51.349365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ede0-e0f2-41fd-bbac-a0d4694ea7fa None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.349644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ede0-e0f2-41fd-bbac-a0d4694ea7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.349917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ede0-e0f2-41fd-bbac-a0d4694ea7fa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.350754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26ede0-e0f2-41fd-bbac-a0d4694ea7fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:51.351905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26ede0-e0f2-41fd-bbac-a0d4694ea7fa None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-0e26ede0-e0f2-41fd-bbac-a0d4694ea7fa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:51.361841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:51.362766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:51.363647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 445/893] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:51.365397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9b1f-2da2-43cc-a6ca-db3895991390 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.365397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9b1f-2da2-43cc-a6ca-db3895991390 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.365397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9b1f-2da2-43cc-a6ca-db3895991390 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.370290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 446/894] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:51.370290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef9b1f-2da2-43cc-a6ca-db3895991390 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.383323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05340833-e328-4434-b757-afd6c7fa2b35 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:51.420357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb14615-2402-4a9f-b375-7ae560e606ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:51.421028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14615-2402-4a9f-b375-7ae560e606ff None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.421313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14615-2402-4a9f-b375-7ae560e606ff None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.421565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb14615-2402-4a9f-b375-7ae560e606ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.432997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb14615-2402-4a9f-b375-7ae560e606ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-0bb14615-2402-4a9f-b375-7ae560e606ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:51.437702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:51.439504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 449/895] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:51.440580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fb306-3155-4ff3-ad9e-a0166b0b139a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.441401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fb306-3155-4ff3-ad9e-a0166b0b139a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.441401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fb306-3155-4ff3-ad9e-a0166b0b139a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.457752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:51.461064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-672fb306-3155-4ff3-ad9e-a0166b0b139a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:51.462267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-672fb306-3155-4ff3-ad9e-a0166b0b139a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:51.486250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef9b1f-2da2-43cc-a6ca-db3895991390 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:51.510296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:51.527578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:51.528187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.528469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.528831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.529466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:51.529765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:51.531131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:51.532783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 450/896] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:51.533830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fb306-3155-4ff3-ad9e-a0166b0b139a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.557570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 447/897] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:51.558477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.661032 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fb306-3155-4ff3-ad9e-a0166b0b139a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:51.663221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722767b-a839-4324-875f-048a0ec0dd66 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:51.699878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:51.700983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.701179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.701438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.702069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:51.702371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:51.703807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:51.704322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:51.705649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.706232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.706814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.708413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:51.709235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:51.711021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:51.714121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 448/898] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:51.714987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.718748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:51.729281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 451/899] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:51.731548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.785595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:51.827360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d0c32-7dd2-44e3-8da1-d362a11aea2c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:51.857203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f63a3f-5399-4ba5-9f45-3f3a1898ded4 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:51.863609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:51.864216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.864487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.865024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.865474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] RBAC: Adding query filter params (interface=public, service_id=614ff7c513634cbf9c37254e3b7dd2aa, region_id=RegionOne) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:51.865772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:51.867132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:51.872894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 449/900] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/endpoints?service_id=614ff7c513634cbf9c37254e3b7dd2aa&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:51.873032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.898127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:51.898751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:51.899029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:51.899295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:51.910485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:51.911926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:51.927655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:51.928684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:51.929788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:51.929788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:51.929788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:51.933441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 452/901] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:51.934326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:51.958490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:51.997877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002447f0-cb52-48fc-ac90-c172a286b699 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:52.032041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:52.036120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:52.036944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:52.037461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:52.037731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:52.038380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:52.039744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:52.040676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134832a-2afd-4d69-8b60-f29335914db1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:52.055145 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 450/902] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:46:51 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:52.056446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:52.105207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:52.105827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:52.106101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:52.106366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:52.106982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:52.108463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:52.118277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 453/903] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:46:51 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:46:52.199255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32ddf6-1e51-45e6-a18f-3549deb439c2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:52.233641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:52.243873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:52.244661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:52.244933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:52.245205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:52.245863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:52.246162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:52.247564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:52.262606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 451/904] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:52.268466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:52.394533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167e077-7b74-47c5-b832-fbd4036d5d5f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:52.436169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:52.437276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:52.437837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:52.438325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:52.447087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:52.448861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:52.461540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:52.462547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:52.463504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:52.466205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 454/905] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:46:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:52.471808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:52.577416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072253df-841f-431a-b13e-ad985c1335fa None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:52.614524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:52.615138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:52.615415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:52.615686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:52.616425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:52.616741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:52.618174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:52.639169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 452/906] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 15:46:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:52.645141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:52.663773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:52.760785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e427c48-bde9-4cf8-813e-993af4bed7fe None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:52.798739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158517e-752f-4a22-a33f-d0f63c106d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:52.799721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158517e-752f-4a22-a33f-d0f63c106d19 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:52.800220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158517e-752f-4a22-a33f-d0f63c106d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:52.800701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158517e-752f-4a22-a33f-d0f63c106d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:52.874558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158517e-752f-4a22-a33f-d0f63c106d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:52.876998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158517e-752f-4a22-a33f-d0f63c106d19 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:52.893688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:52.992619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 455/907] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:46:52 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:46:52.998106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.034303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a707f-808b-47fb-bc75-46446af9a68e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.034630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a707f-808b-47fb-bc75-46446af9a68e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.034846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a707f-808b-47fb-bc75-46446af9a68e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.036481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 456/908] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:53.042011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88809d30-74a6-4320-80e8-443f65522c8c None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.042287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88809d30-74a6-4320-80e8-443f65522c8c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.042557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88809d30-74a6-4320-80e8-443f65522c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.057073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:53.063954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-88809d30-74a6-4320-80e8-443f65522c8c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:53.065199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-88809d30-74a6-4320-80e8-443f65522c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:53.114557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b7853-aa18-4973-9f1a-ae25f3ba4bce None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.142344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 457/909] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:53.150195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.150833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.151107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.151373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.163392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.164901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:53.181115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:53.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:53.183045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:53.183045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:53.183045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:53.186878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 453/910] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:53.188228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88809d30-74a6-4320-80e8-443f65522c8c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.239693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:53.295313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88809d30-74a6-4320-80e8-443f65522c8c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.337635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.338376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.338665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.338938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.339591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:53.340089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.342284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.355561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 458/911] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:53.361430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.408132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.468837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:53.475208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409ea28-f3c5-4041-a81b-804b3eea263b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.512484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.513498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.514136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.514600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.522864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.524513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd643c-ea6f-4866-a6a5-cdd2e57a40d8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.524896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.527014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:53.539831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:53.540852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:53.541938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:53.543953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 454/912] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:53.552505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.562425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.563068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.563372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.563797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.564403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.565823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:53.576627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:53.577837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:53.578807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 459/913] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:53.586923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.680666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9c344-e719-4807-95e0-7a3d8487ad1c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.708847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c059661-388e-4d5a-a9b5-79c9834dcb92 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.718744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.720167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.720642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.721153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.722064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:53.722735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.725427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.739499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 455/914] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:53.744695 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.745353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.745461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.745738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.746110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.746808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:53.747182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.748808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.759250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 460/915] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:53.760270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:53.773511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:53.806297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:53.872837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f399f-2c6e-49e6-8b44-f806042225b5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.882335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a57ca-27cf-4a6c-842a-1848cb51e7fa None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:53.914916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.915486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.915767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.916038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.916793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] RBAC: Adding query filter params (role.id=f41f57df30b84ba5ae1b0472240664e7, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:53.917094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.918530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.920412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64a1f4-d3f7-4f15-8d75-9553d7da47c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:53.921310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64a1f4-d3f7-4f15-8d75-9553d7da47c3 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:53.921783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64a1f4-d3f7-4f15-8d75-9553d7da47c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:53.922332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64a1f4-d3f7-4f15-8d75-9553d7da47c3 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:53.923439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64a1f4-d3f7-4f15-8d75-9553d7da47c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:53.924010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64a1f4-d3f7-4f15-8d75-9553d7da47c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:53.925059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 456/916] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:46:53 2022] GET /identity/v3/role_assignments?user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=f41f57df30b84ba5ae1b0472240664e7 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:53.926062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64a1f4-d3f7-4f15-8d75-9553d7da47c3 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:53.935691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 461/917] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Oct 3 15:46:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:53.942494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:54.043511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:54.068217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba429cb-14a4-4a63-9ab8-1af1e21e3a9c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:54.108397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823356c-e546-4b36-ac78-e54bbbdac83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:54.109542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823356c-e546-4b36-ac78-e54bbbdac83e None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:54.109542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823356c-e546-4b36-ac78-e54bbbdac83e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:54.109542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823356c-e546-4b36-ac78-e54bbbdac83e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:54.110132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823356c-e546-4b36-ac78-e54bbbdac83e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:54.112057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823356c-e546-4b36-ac78-e54bbbdac83e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:54.135323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 457/918] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:46:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:54.479854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b0f3-abb3-4803-a460-f1db3f11fcb6 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:54.480208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b0f3-abb3-4803-a460-f1db3f11fcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:54.480443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b0f3-abb3-4803-a460-f1db3f11fcb6 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:54.482105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 462/919] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:54.488147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5250fcc-b33b-457f-bbc4-7ae6fa29f7ba None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:54.488434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5250fcc-b33b-457f-bbc4-7ae6fa29f7ba None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:54.488713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5250fcc-b33b-457f-bbc4-7ae6fa29f7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:54.507686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e5250fcc-b33b-457f-bbc4-7ae6fa29f7ba None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:54.508915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e5250fcc-b33b-457f-bbc4-7ae6fa29f7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:54.530157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd764-55d7-4ba5-9086-96d7fb934cc0 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:54.530488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd764-55d7-4ba5-9086-96d7fb934cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:54.530707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd764-55d7-4ba5-9086-96d7fb934cc0 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:54.532884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 463/920] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:54.537789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b579-228a-4fd2-892e-e030220adb67 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:54.538076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b579-228a-4fd2-892e-e030220adb67 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:54.538352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1b579-228a-4fd2-892e-e030220adb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:54.561953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a9a1b579-228a-4fd2-892e-e030220adb67 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:54.563665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a9a1b579-228a-4fd2-892e-e030220adb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:54.588472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 458/921] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:54.659122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 464/922] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:54.897436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5250fcc-b33b-457f-bbc4-7ae6fa29f7ba None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:54.963195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:54.967806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b579-228a-4fd2-892e-e030220adb67 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.000485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:55.003196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5250fcc-b33b-457f-bbc4-7ae6fa29f7ba None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.038771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.039305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.039584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.039852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.040468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.042019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:55.052522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:55.053983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:55.054829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:55.058047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 459/923] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:55.059317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.089381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1b579-228a-4fd2-892e-e030220adb67 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.132057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.133243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.133824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.134317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.135133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.136955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:55.147920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:55.149142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 15:46:55.150035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:55.158166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 465/924] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Oct 3 15:46:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:55.159375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.169838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f3115-8f9b-4b7d-adb2-5ebf395c6310 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.199554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:55.207443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.208861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.209032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.209403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.210139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:55.210546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.212123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.223227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 460/925] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:46:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:55.224841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.264698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:55.276794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd1d56-62a3-477e-9f8e-fe69700d5292 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.323931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.324751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.325179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.325720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.326543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:55.327027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.328698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.338956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e68c5-8e4f-4a12-80c0-746f49419bf8 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.339244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 466/926] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:55.340401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.376289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.377364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.377715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.378057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.382566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:55.382931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.384524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.396713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 461/927] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:46:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:55.398505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.456093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015ea0f-3b5a-4fc9-8bb0-171a2bbcdd40 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.460101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:55.500544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.501406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.501676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.501984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.502694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:55.502998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.504480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.510137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 467/928] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:55.511280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf01b7-72a3-4b52-9c3b-e5c58595ab7c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.520813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.544357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:55.552159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.552803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.553191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.553464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.554193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] RBAC: Adding query filter params (interface=public, service_id=8bb47df689d9413d93d2f54794a51b71, region_id=RegionOne) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:55.554500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.555994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.566548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 462/929] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:46:55 2022] GET /identity/v3/endpoints?service_id=8bb47df689d9413d93d2f54794a51b71&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:55.567671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:55.643515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be5f87-1362-4299-80f4-d3d3376b349d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.684970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c9f3c-1ea8-4f9a-b4a3-e298d2f9fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.685951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c9f3c-1ea8-4f9a-b4a3-e298d2f9fd98 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.686436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c9f3c-1ea8-4f9a-b4a3-e298d2f9fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.686909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c9f3c-1ea8-4f9a-b4a3-e298d2f9fd98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.688912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c9f3c-1ea8-4f9a-b4a3-e298d2f9fd98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.691246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c9f3c-1ea8-4f9a-b4a3-e298d2f9fd98 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.700612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d0f5f-e301-40f6-a117-93e9d40b944c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:55.729418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 468/930] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:46:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:46:55.731378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:55.755204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72f43d-874d-4e61-b17d-69439f0361ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:55.756427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72f43d-874d-4e61-b17d-69439f0361ed None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:55.757029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72f43d-874d-4e61-b17d-69439f0361ed None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:55.757587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72f43d-874d-4e61-b17d-69439f0361ed None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:55.758792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72f43d-874d-4e61-b17d-69439f0361ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:55.760820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72f43d-874d-4e61-b17d-69439f0361ed None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:55.769036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 463/931] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:46:55 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:46:56.199842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf872a-e414-4358-86a3-4011656fe2cb None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:56.200393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf872a-e414-4358-86a3-4011656fe2cb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:56.200393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf872a-e414-4358-86a3-4011656fe2cb None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:56.202028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 469/932] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:56.207282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237fe1-c4ae-4cef-8d02-db495e19d84c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:56.207681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237fe1-c4ae-4cef-8d02-db495e19d84c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:56.207867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237fe1-c4ae-4cef-8d02-db495e19d84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:56.227792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ec237fe1-c4ae-4cef-8d02-db495e19d84c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:56.229031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ec237fe1-c4ae-4cef-8d02-db495e19d84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:56.307824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 464/933] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:56.411390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aac27-49f2-4f22-bfe4-0131f117ae85 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:56.411670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aac27-49f2-4f22-bfe4-0131f117ae85 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:56.411946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aac27-49f2-4f22-bfe4-0131f117ae85 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:56.413583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 470/934] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:56.422165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b3cb1-6dbd-4fe2-8e91-ea5d5d0c9aaf None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:56.422445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b3cb1-6dbd-4fe2-8e91-ea5d5d0c9aaf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:56.423345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b3cb1-6dbd-4fe2-8e91-ea5d5d0c9aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:56.444782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-af4b3cb1-6dbd-4fe2-8e91-ea5d5d0c9aaf None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:56.446365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-af4b3cb1-6dbd-4fe2-8e91-ea5d5d0c9aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:56.521911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 465/935] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:56.581586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aac27-49f2-4f22-bfe4-0131f117ae85 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:56.661177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:56.687789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aac27-49f2-4f22-bfe4-0131f117ae85 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:56.726482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:56.727102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:56.727375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:56.727642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:56.738164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:46:56.743378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:56.744307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:56.745189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 471/936] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 15:46:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:56.751467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b3cb1-6dbd-4fe2-8e91-ea5d5d0c9aaf None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:56.795930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:56.803759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:56.866549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b3cb1-6dbd-4fe2-8e91-ea5d5d0c9aaf None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:56.909477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:56.910346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:56.910721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:56.911090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:56.911948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:56.912351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:56.913942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:56.925430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 466/937] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 15:46:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:56.926991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287694c-ce44-455e-b542-0b397d82435d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:56.936341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:56.963435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:56.964047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:56.964326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:56.964599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:56.965284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:56.966861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:56.970482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING keystone.server.flask.common [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 03 15:46:56.974494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:56.991831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.sql.core [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 15:46:56.991831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 15:46:56.991831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [parameters: {'id': '0347ad2c81464157a7ef1bdebf47d3e8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 15:46:56.991831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81367) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0347ad2c81464157a7ef1bdebf47d3e8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0347ad2c81464157a7ef1bdebf47d3e8. Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application conn.query(q) Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 15:46:56.999759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result.read() Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 03 15:46:57.000805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application next(self.gen) Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application yield resource Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application next(self.gen) Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise self.value Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application session.commit() Oct 03 15:46:57.001813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.session.flush() Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application compat.raise_( Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 15:46:57.002697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 15:46:57.003573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application conn.query(q) Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result.read() Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 15:46:57.004376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [parameters: {'id': '0347ad2c81464157a7ef1bdebf47d3e8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:57.005269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:57.006052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0347ad2c81464157a7ef1bdebf47d3e8. Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 472/938] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Oct 3 15:46:56 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:57.008639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:57.073800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539059f-2c2a-4bd0-8056-9992d7779898 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:57.117908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:57.122587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:57.123486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:57.123962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:57.124423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:57.124679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11db970-5c3e-4355-8bf8-b9c8e32769f5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:57.139261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:57.140962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:57.163926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:57.164959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:57.165871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:57.165871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:57.165871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:57.165871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 467/939] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:57.171785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:57.193021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:57.194022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:57.194368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:57.194705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:57.205199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:57.207837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:57.223968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:57.224911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:57.225821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:57.228801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 473/940] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:46:57.230814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:57.292174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:57.303781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc117b-b22d-40f2-889a-b0e2731f30d2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:57.349586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:57.351277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:57.351767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:57.352493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:57.353798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:57.354309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:57.355910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:57.371573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516e08d-6ace-4015-8d1e-2b9354f63dbc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:57.373670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 468/941] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:57.381746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:57.397942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:57.410858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:57.411787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:57.412271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:57.412798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:57.413670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:57.414176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:57.415977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:57.437634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 474/942] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:57.496304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e4f54-40cd-4a6a-ba02-48629a3f24d0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:57.536990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:57.537635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:57.537908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:57.538173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:57.546577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:57.548195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:46:57.561264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:57.562307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:57.563849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 469/943] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:46:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:57.569508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:57.692036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2f3bd-c5f9-49ef-9f41-961189fc9a2d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:57.703736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:57.731541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:57.732151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:57.732423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:57.732687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:57.733455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:57.733753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:57.735739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:57.754949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 475/944] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 15:46:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:57.760319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:57.840060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:57.867752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b90fe-bac8-4adc-a8b2-d5ba219afc0f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:57.908570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a6b88-5cff-4e4d-9d21-8e1a3de37044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:57.909326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a6b88-5cff-4e4d-9d21-8e1a3de37044 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:57.909605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a6b88-5cff-4e4d-9d21-8e1a3de37044 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:57.909874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a6b88-5cff-4e4d-9d21-8e1a3de37044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:57.921919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a6b88-5cff-4e4d-9d21-8e1a3de37044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:57.923929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a6b88-5cff-4e4d-9d21-8e1a3de37044 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-379a6b88-5cff-4e4d-9d21-8e1a3de37044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:46:57.939382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:57.940342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:46:57.941257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:46:57.941257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:46:57.941257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:46:57.944895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 470/945] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:57.946476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.007648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea580f8-22a5-490a-8650-35f28f97297a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.007976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea580f8-22a5-490a-8650-35f28f97297a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.008250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea580f8-22a5-490a-8650-35f28f97297a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.009850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 471/946] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:58.015475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f82a7-2e10-4888-9c1d-ef48d94c22a5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.015755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f82a7-2e10-4888-9c1d-ef48d94c22a5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.016024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f82a7-2e10-4888-9c1d-ef48d94c22a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.041539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-5e8f82a7-2e10-4888-9c1d-ef48d94c22a5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:58.042861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5e8f82a7-2e10-4888-9c1d-ef48d94c22a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:58.064682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd60ee7-2d60-4adf-8ade-b95b33d11594 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:58.106299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:58.106921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.107200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.107482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.108135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:58.108436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:58.110655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:58.127205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 476/947] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:58.133252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.141638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 472/948] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:46:58.166304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18578d-f640-4793-86d9-1f7e66e359d4 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.166750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18578d-f640-4793-86d9-1f7e66e359d4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.167033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18578d-f640-4793-86d9-1f7e66e359d4 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.169266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 473/949] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:46:58.169768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:58.176510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b43dd-c7b8-43ef-b34d-79fdd9542a0b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.177335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b43dd-c7b8-43ef-b34d-79fdd9542a0b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.177857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b43dd-c7b8-43ef-b34d-79fdd9542a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.202254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a09b43dd-c7b8-43ef-b34d-79fdd9542a0b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:58.204136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a09b43dd-c7b8-43ef-b34d-79fdd9542a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:58.257897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e10a7-39f5-4be8-b8a9-456566134c50 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:58.266749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:58.293133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 474/950] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:46:58.310413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:58.311358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.311820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.312267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.326574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:58.328260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:46:58.340843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:58.341737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:46:58.342625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:58.344774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 477/951] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:58.352765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b43dd-c7b8-43ef-b34d-79fdd9542a0b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.491072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b43dd-c7b8-43ef-b34d-79fdd9542a0b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:58.496215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.529058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:58.529665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.529937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.530202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.530925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:58.531223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:58.532683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:58.546691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 475/952] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:46:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:58.552960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.614260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:58.615203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:58.628173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553286ab-7786-40a6-96ed-0b6e1b544778 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:58.665010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:58.665638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.665912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.666178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.666761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:58.668203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:58.678221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:58.679248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:58.680330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 478/953] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:46:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:58.681262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.687029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ffb04-e5eb-4d7c-95e2-156859e13091 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:58.735274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:58.735935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.736205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.736472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/cd0c6adf0bb34ba0b33bad9d46b47ac8/roles/f41f57df30b84ba5ae1b0472240664e7` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.738705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=cd0c6adf0bb34ba0b33bad9d46b47ac8, role_id=f41f57df30b84ba5ae1b0472240664e7)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:58.740140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:58.756905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 476/954] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:46:58 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/cd0c6adf0bb34ba0b33bad9d46b47ac8/roles/f41f57df30b84ba5ae1b0472240664e7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:46:58.757666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.830462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7a82b-3d2b-475b-b522-8cec3d9feb94 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:58.894386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d850af3-7dd7-4cfb-aaac-e8ff9a00d3da None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:58.901294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:58.902556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.903164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.903645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.905176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:58.905408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:58.909483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:46:58.927627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:46:58.928948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:46:58.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:46:58.935389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 479/955] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:46:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:46:58.939401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:58.960012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:58.960643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:58.961040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:58.961316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:58.962614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:58.962917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:58.964626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:58.965987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:58.973878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 477/956] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:46:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:58.979425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:59.086462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b419b86-834f-4635-8de8-7da0b7425868 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:59.121398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257eed8-be1d-4d63-a0f9-ee9f24096fdb None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:59.128206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:59.128820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:59.129197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:59.129589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:59.130324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:59.130726 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:59.132216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:59.138079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 480/957] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:46:59.144496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:59.169852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:59.170716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:59.171096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:59.171485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:59.172375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:59.172990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:59.175754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:59.184088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 478/958] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:46:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:59.191206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:46:59.230138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:59.253178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:46:59.265795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264324f-21c8-440f-a21a-8b8b902a95a9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:59.312245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7bade-c171-4588-8ba3-4da12ff4d651 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:46:59.324591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332b71-4c29-4dce-8743-59555fefe103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:59.325551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332b71-4c29-4dce-8743-59555fefe103 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:59.326033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332b71-4c29-4dce-8743-59555fefe103 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:59.327060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332b71-4c29-4dce-8743-59555fefe103 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:59.328328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a332b71-4c29-4dce-8743-59555fefe103 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:46:59.329809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a332b71-4c29-4dce-8743-59555fefe103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:59.332521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a332b71-4c29-4dce-8743-59555fefe103 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:59.340123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 481/959] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:46:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:46:59.363680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ecece-c342-425b-9ca0-62914504fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:46:59.364490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecece-c342-425b-9ca0-62914504fc04 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:59.364904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecece-c342-425b-9ca0-62914504fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:59.365275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ecece-c342-425b-9ca0-62914504fc04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:59.366090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecece-c342-425b-9ca0-62914504fc04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:46:59.367619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ecece-c342-425b-9ca0-62914504fc04 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:46:59.394192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 479/960] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:46:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:46:59.841307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a581d-52a2-432f-8563-a9069a7f5d53 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:59.841614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a581d-52a2-432f-8563-a9069a7f5d53 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:59.841981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a581d-52a2-432f-8563-a9069a7f5d53 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:59.843715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 482/961] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:46:59.852061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a397272-9fb7-4b15-a074-9a89fbc1426b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:46:59.852532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a397272-9fb7-4b15-a074-9a89fbc1426b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:46:59.852884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a397272-9fb7-4b15-a074-9a89fbc1426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:46:59.873654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-1a397272-9fb7-4b15-a074-9a89fbc1426b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:46:59.874868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1a397272-9fb7-4b15-a074-9a89fbc1426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:46:59.964399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 480/962] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:46:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:00.239547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a581d-52a2-432f-8563-a9069a7f5d53 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:00.355132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a581d-52a2-432f-8563-a9069a7f5d53 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:00.396622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:00.397651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:00.398133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:00.398657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:00.399581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:00.401256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:00.417404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.sql.core [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 15:47:00.417404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 15:47:00.417404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [parameters: {'user_id': 'b2dde23fe6454f7fab71adef81b33f8f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 15:47:00.417404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81367) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application conn.query(q) Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result.read() Oct 03 15:47:00.425468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 15:47:00.426359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application next(self.gen) Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application yield resource Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application next(self.gen) Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise self.value Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application session.commit() Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 15:47:00.427300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.session.flush() Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application compat.raise_( Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 15:47:00.428187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 15:47:00.429148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application conn.query(q) Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application result.read() Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 15:47:00.430107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b2dde23fe6454f7fab71adef81b33f8f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:00.430968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 03 15:47:00.431782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 03 15:47:00.433980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 03 15:47:00.433980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 03 15:47:00.433980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 15:47:00.433980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 15:47:00.433980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 15:47:00.433980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:00.436385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 483/963] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Oct 3 15:47:00 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:00.442201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a397272-9fb7-4b15-a074-9a89fbc1426b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:00.461482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:00.548234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a397272-9fb7-4b15-a074-9a89fbc1426b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:00.583675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b0b70-0af5-4f58-b61b-eec1fa9c376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:00.584304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0b70-0af5-4f58-b61b-eec1fa9c376a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:00.584575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0b70-0af5-4f58-b61b-eec1fa9c376a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:00.584872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0b70-0af5-4f58-b61b-eec1fa9c376a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:00.592254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b0b70-0af5-4f58-b61b-eec1fa9c376a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:00.593709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b0b70-0af5-4f58-b61b-eec1fa9c376a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-627b0b70-0af5-4f58-b61b-eec1fa9c376a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:47:00.605716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:00.606667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:00.607598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 481/964] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:47:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:00.613086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:00.623553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:00.729904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007ad52-f757-4bc7-9574-bfddc98a8d1c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:00.767772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42004f-49cd-475a-b1bd-51d6408cfccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:00.768893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42004f-49cd-475a-b1bd-51d6408cfccc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:00.769401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42004f-49cd-475a-b1bd-51d6408cfccc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:00.769870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42004f-49cd-475a-b1bd-51d6408cfccc None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:00.770787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42004f-49cd-475a-b1bd-51d6408cfccc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:00.771290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42004f-49cd-475a-b1bd-51d6408cfccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:00.773186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42004f-49cd-475a-b1bd-51d6408cfccc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:00.813545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 484/965] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Oct 3 15:47:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:01.234210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5250d-7498-4028-89cb-1b5cfec83578 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.234383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5250d-7498-4028-89cb-1b5cfec83578 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.234717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5250d-7498-4028-89cb-1b5cfec83578 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.236070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 482/966] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:01.241450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afa838-71da-4bef-887e-7e92f4adf3ad None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.241742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afa838-71da-4bef-887e-7e92f4adf3ad None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.242018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afa838-71da-4bef-887e-7e92f4adf3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.261326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-13afa838-71da-4bef-887e-7e92f4adf3ad None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:01.262845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-13afa838-71da-4bef-887e-7e92f4adf3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:01.332116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:01.341331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 485/967] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:01.613008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5250d-7498-4028-89cb-1b5cfec83578 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:01.663098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29e12-369a-43f3-84eb-88dbbbc26ac6 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.663630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29e12-369a-43f3-84eb-88dbbbc26ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.663630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29e12-369a-43f3-84eb-88dbbbc26ac6 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.665032 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 486/968] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:01.670399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34f1d5-36ce-44e6-9e3f-9340b3b01a4e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.670679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34f1d5-36ce-44e6-9e3f-9340b3b01a4e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.670951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34f1d5-36ce-44e6-9e3f-9340b3b01a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.690638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2e34f1d5-36ce-44e6-9e3f-9340b3b01a4e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:01.691821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2e34f1d5-36ce-44e6-9e3f-9340b3b01a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:01.699434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:01.726827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5250d-7498-4028-89cb-1b5cfec83578 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:01.760570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:01.760942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 487/969] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:01.761222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.761500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.761769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.761933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f3f89-b2ca-4d40-8350-ea979dd652bd None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.762175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f3f89-b2ca-4d40-8350-ea979dd652bd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.762448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f3f89-b2ca-4d40-8350-ea979dd652bd None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.763959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 488/970] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:01.769738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e089a-d71b-41cd-ae7f-193ca842dc39 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.770015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e089a-d71b-41cd-ae7f-193ca842dc39 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.770283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e089a-d71b-41cd-ae7f-193ca842dc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.778818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:01.784676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:01.785777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:01.786815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:01.788825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-843e089a-d71b-41cd-ae7f-193ca842dc39 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:01.789207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 483/971] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 15:47:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:01.789963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-843e089a-d71b-41cd-ae7f-193ca842dc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:01.794659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:01.864350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 489/972] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:01.906534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c834e56-962a-4023-b9c1-739484a21bbf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:01.942728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:01.943338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:01.943616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:01.943885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:01.944549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:01.944879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:01.946245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:01.948764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:01.960612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 484/973] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 15:47:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:01.961875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e089a-d71b-41cd-ae7f-193ca842dc39 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.021415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:02.054297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.069876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e089a-d71b-41cd-ae7f-193ca842dc39 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.106537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.107170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.107449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.107720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.118740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.120146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:02.136454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:02.137484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:02.138375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:02.138375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:02.138375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:02.138375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 490/974] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:02.141973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.162325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dd654-eae3-4552-8002-922492e9cc89 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.195729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.196334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.196612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.196913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.197568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.198940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.213535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 485/975] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:47:02 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:47:02.214335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.252684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d56b2-691c-416f-9c3b-0a7a43f4e5e3 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.262304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:02.287485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:02.310052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.311039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.311587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.312040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.313749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.315990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:02.330043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:02.331134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:02.332595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 491/976] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:02.337381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.345076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da6738-9d00-4a1e-be93-33eca85d75bb None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.386258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.387111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.387393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.387664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.388317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:02.388621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.390033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.404077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 486/977] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:02.410416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.449026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff46f4-1d85-4250-a566-8ce552a1f66f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.482136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.482739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.483012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.483282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.483906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:02.484206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.486126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.490961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 492/978] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:02.496514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.523925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a5e14-0722-4e3e-b253-90d09e076930 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.530804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:02.545607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:02.566158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.566792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.567073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.567336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.576253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.577740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:47:02.590976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:02.591933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:02.592921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 487/979] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:02.598176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.606997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e389417-0a2d-4a5d-b760-64c027f51269 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.643128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.643677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.643953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.644225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.644895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:02.645198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.646594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.651925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 493/980] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:02.658050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.715070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bfd8e-f8f7-485c-9947-796c10eb1cba None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.751971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.752624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.752935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.753214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.753945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:02.754249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.755682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.767841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:02.774500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 488/981] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:02.778636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7dff-c896-4d3c-9a57-31674363d901 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.779848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.816523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.817156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.817431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.817695 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.818605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] RBAC: Adding query filter params (interface=admin, service_id=8bb47df689d9413d93d2f54794a51b71, region_id=RegionOne) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:02.818719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.819304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:02.820187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.826345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 494/982] 10.222.0.33 () {56 vars in 1249 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/endpoints?service_id=8bb47df689d9413d93d2f54794a51b71&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:02.832213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:02.888417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3dd01-3f84-436f-a68f-cc5c4b67e8e0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.924355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d0891-a48a-427c-9518-31122120e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.924912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0891-a48a-427c-9518-31122120e09b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.925188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0891-a48a-427c-9518-31122120e09b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.925456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d0891-a48a-427c-9518-31122120e09b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.943753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a97758-1ffd-40d5-a992-379cacc1db98 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:02.983637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:02.984506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:02.985355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:02.985639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:02.986335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:02.987739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:02.994059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 495/983] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:47:03.013277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d0891-a48a-427c-9518-31122120e09b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:03.014776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d0891-a48a-427c-9518-31122120e09b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:03.053595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:03.128858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 489/984] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:02 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 15:47:03.134930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:03.161725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:03.247456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0091c35-d99a-4b0d-a6b8-0c7795615968 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:03.283517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:03.284153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:03.284433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:03.284703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:03.295364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:03.296850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:03.317278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:03.318261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:03.319158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:03.319158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:03.319158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:03.323070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 496/985] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:03.324490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d0891-a48a-427c-9518-31122120e09b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:03.427002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d0891-a48a-427c-9518-31122120e09b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:03.462038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:03.462641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:03.462999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:03.463267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:03.463906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:03.464203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:03.465598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:03.468020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:03.480269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 490/986] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:03.486522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:03.574649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:03.601585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfeb32-df9c-4d7f-ae5f-a4c70ca08530 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:03.635829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:03.636452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:03.636761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:03.637019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:03.643806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:03.645287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:47:03.656389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:03.658289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:03.659183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 497/987] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:47:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:03.665518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:03.771369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97fed7-638f-4d79-bb84-748afe5618fc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:03.817352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d67ca-d1cd-49ef-93b6-aad006da492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:03.817967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d67ca-d1cd-49ef-93b6-aad006da492a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:03.818242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d67ca-d1cd-49ef-93b6-aad006da492a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:03.818509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d67ca-d1cd-49ef-93b6-aad006da492a None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:03.819205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d67ca-d1cd-49ef-93b6-aad006da492a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:03.819506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d67ca-d1cd-49ef-93b6-aad006da492a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:03.820954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d67ca-d1cd-49ef-93b6-aad006da492a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:03.835290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 491/988] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:47:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:03.841006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:03.950098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3ae84-a8ca-47c2-8927-26ac3583300b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:03.986173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce311722-58c4-48ea-b69b-4fa2e2bc740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:03.986776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce311722-58c4-48ea-b69b-4fa2e2bc740e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:03.987048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce311722-58c4-48ea-b69b-4fa2e2bc740e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:03.987318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce311722-58c4-48ea-b69b-4fa2e2bc740e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:03.988044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce311722-58c4-48ea-b69b-4fa2e2bc740e None admin] RBAC: Adding query filter params (role.id=f41f57df30b84ba5ae1b0472240664e7, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:03.988353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce311722-58c4-48ea-b69b-4fa2e2bc740e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:03.989820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce311722-58c4-48ea-b69b-4fa2e2bc740e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:03.991617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:03.997358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 498/989] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:03 2022] GET /identity/v3/role_assignments?user.id=cd0c6adf0bb34ba0b33bad9d46b47ac8&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=f41f57df30b84ba5ae1b0472240664e7 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:04.436633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa267-2d0a-41a0-ae4a-5b77f0079bda None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:04.437015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa267-2d0a-41a0-ae4a-5b77f0079bda None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:04.437219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa267-2d0a-41a0-ae4a-5b77f0079bda None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:04.438764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 492/990] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:04.443917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a2f2-cb82-4a8d-9371-1b38995a9964 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:04.444208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a2f2-cb82-4a8d-9371-1b38995a9964 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:04.444480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2a2f2-cb82-4a8d-9371-1b38995a9964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:04.462609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-60c2a2f2-cb82-4a8d-9371-1b38995a9964 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:04.463799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-60c2a2f2-cb82-4a8d-9371-1b38995a9964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:04.533429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 499/991] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:04.799406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa267-2d0a-41a0-ae4a-5b77f0079bda None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:04.859908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:04.907822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa267-2d0a-41a0-ae4a-5b77f0079bda None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:04.943768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:04.944401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:04.944679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:04.944973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:04.945552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:04.946928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:04.955884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:04.956887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:47:04.957730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:04.960801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 493/992] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:47:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:04.963197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2a2f2-cb82-4a8d-9371-1b38995a9964 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:05.064243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2a2f2-cb82-4a8d-9371-1b38995a9964 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:05.094797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:05.104239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0eb72-fec1-42a7-9c5e-d898208832a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:05.104968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0eb72-fec1-42a7-9c5e-d898208832a9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.105312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0eb72-fec1-42a7-9c5e-d898208832a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.105647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0eb72-fec1-42a7-9c5e-d898208832a9 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.106355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0eb72-fec1-42a7-9c5e-d898208832a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:05.106727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0eb72-fec1-42a7-9c5e-d898208832a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:05.108179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0eb72-fec1-42a7-9c5e-d898208832a9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:05.118619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 500/993] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:47:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:05.119746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:05.190790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86928b2-11d6-4bed-8d9a-c19eb0938063 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.191118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86928b2-11d6-4bed-8d9a-c19eb0938063 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.191347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86928b2-11d6-4bed-8d9a-c19eb0938063 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.192773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 501/994] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:47:05.197519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1be91-d85d-4c1d-bf81-a655384f5205 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.197795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1be91-d85d-4c1d-bf81-a655384f5205 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.198061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1be91-d85d-4c1d-bf81-a655384f5205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.217295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-aca1be91-d85d-4c1d-bf81-a655384f5205 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:05.218452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-aca1be91-d85d-4c1d-bf81-a655384f5205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:05.223371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf33824-89cb-4ee5-9f29-f71cdde66f40 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:05.243804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:05.257993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:05.259137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.259137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.259137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.259725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:05.260021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:05.261421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:05.266848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 494/995] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:47:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:05.273426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:05.296783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 502/996] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:05.386004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee694e-0e1d-47fb-b020-faad3f9935e4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:05.419682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:05.420409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.420838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.421205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.422005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] RBAC: Adding query filter params (interface=public, service_id=4d0fb6cd65824a299cda7ced243b10cb, region_id=RegionOne) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:05.423044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:05.424652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:05.429863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 495/997] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:47:05 2022] GET /identity/v3/endpoints?service_id=4d0fb6cd65824a299cda7ced243b10cb&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:05.435322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1be91-d85d-4c1d-bf81-a655384f5205 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:05.536199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1be91-d85d-4c1d-bf81-a655384f5205 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:05.559409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:05.571854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:05.572399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.572677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.572974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.573584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:05.574994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:05.577425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:05.588972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 503/998] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:47:05 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:47:05.633927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:05.691539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1df30-6ee5-4500-b8e6-c5d588c810b5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:05.728881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:05.729692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.730096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.730504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.731231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:05.732794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:05.742495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:05.743554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:05.745747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 496/999] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Oct 3 15:47:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:05.751769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:05.856328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2f8d3-912c-4116-87c6-2e6d00b6cb70 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:05.894623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:05.895229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:05.895502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:05.895765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:05.896385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:05.896682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:05.898076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:05.908300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 504/1000] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:47:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:05.909456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:06.011815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af629cb5-ed04-4214-8ae1-9445df8e8b75 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:06.025436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:06.048314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87e82a-8624-4949-af9e-808fa5d23e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:06.048945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87e82a-8624-4949-af9e-808fa5d23e2b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:06.049222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87e82a-8624-4949-af9e-808fa5d23e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:06.049488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87e82a-8624-4949-af9e-808fa5d23e2b None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:06.050127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87e82a-8624-4949-af9e-808fa5d23e2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:06.050400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87e82a-8624-4949-af9e-808fa5d23e2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:06.051759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87e82a-8624-4949-af9e-808fa5d23e2b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:06.061227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 497/1001] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Oct 3 15:47:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:06.062673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:06.119409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:06.176736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb19784-44cf-4f76-901d-a560f88d300c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:06.210966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50421739-590c-4664-bd87-e9c60014baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:06.211570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50421739-590c-4664-bd87-e9c60014baa6 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:06.211849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50421739-590c-4664-bd87-e9c60014baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:06.212118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50421739-590c-4664-bd87-e9c60014baa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:06.212823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50421739-590c-4664-bd87-e9c60014baa6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:06.214290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50421739-590c-4664-bd87-e9c60014baa6 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:06.238723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79326a0-bcaf-43d8-b6b4-eff9c19ff6ad None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:06.239021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79326a0-bcaf-43d8-b6b4-eff9c19ff6ad None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:06.239305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79326a0-bcaf-43d8-b6b4-eff9c19ff6ad None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:06.244848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 505/1002] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:47:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:47:06.245038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 498/1003] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:47:06.246340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a6eec-b701-496f-88e7-bc845bbc80d8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:06.246630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a6eec-b701-496f-88e7-bc845bbc80d8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:06.246956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a6eec-b701-496f-88e7-bc845bbc80d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:06.267503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-371a6eec-b701-496f-88e7-bc845bbc80d8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:06.268766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-371a6eec-b701-496f-88e7-bc845bbc80d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:06.361139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 506/1004] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:06.629218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79326a0-bcaf-43d8-b6b4-eff9c19ff6ad None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:06.732224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79326a0-bcaf-43d8-b6b4-eff9c19ff6ad None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:06.766552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:06.767287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:06.767565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:06.767834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] PATH_INFO: `/v3/services/image` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:06.768474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:06.769881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:06.778693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:06.779675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:06.780497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:06.785088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 499/1005] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Oct 3 15:47:06 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:06.786352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a6eec-b701-496f-88e7-bc845bbc80d8 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:06.821769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:06.891581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a6eec-b701-496f-88e7-bc845bbc80d8 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:06.928138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91884c-c340-4021-ac5d-6dbd77c8497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:06.929125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91884c-c340-4021-ac5d-6dbd77c8497b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:06.929588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91884c-c340-4021-ac5d-6dbd77c8497b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:06.930065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91884c-c340-4021-ac5d-6dbd77c8497b None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:06.930878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91884c-c340-4021-ac5d-6dbd77c8497b None admin] RBAC: Adding query filter params (name=image) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:06.931362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91884c-c340-4021-ac5d-6dbd77c8497b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:06.932965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91884c-c340-4021-ac5d-6dbd77c8497b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:06.942952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 507/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:06.944535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:06.989381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:07.049496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30f341-e089-44f8-a09a-667a996b77cb None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:07.082715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a798fb-18a0-4b7c-81d6-607d8c0e36cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:07.083311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a798fb-18a0-4b7c-81d6-607d8c0e36cf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:07.083583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a798fb-18a0-4b7c-81d6-607d8c0e36cf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:07.083849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a798fb-18a0-4b7c-81d6-607d8c0e36cf None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:07.084468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a798fb-18a0-4b7c-81d6-607d8c0e36cf None admin] RBAC: Adding query filter params (type=image) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:07.084785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a798fb-18a0-4b7c-81d6-607d8c0e36cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:07.086148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a798fb-18a0-4b7c-81d6-607d8c0e36cf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:07.090671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 500/1007] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:07.775004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af921ff8-9064-4f12-86f5-cf3a47c74625 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:07.775538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af921ff8-9064-4f12-86f5-cf3a47c74625 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:07.775538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af921ff8-9064-4f12-86f5-cf3a47c74625 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:07.776878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 508/1008] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:07.782017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee0efe-cc53-4cfb-bfd7-c70013d2aa62 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:07.782303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee0efe-cc53-4cfb-bfd7-c70013d2aa62 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:07.782567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee0efe-cc53-4cfb-bfd7-c70013d2aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:07.801465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-75ee0efe-cc53-4cfb-bfd7-c70013d2aa62 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:07.802661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-75ee0efe-cc53-4cfb-bfd7-c70013d2aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:07.877955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 501/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:07 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:08.151195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af921ff8-9064-4f12-86f5-cf3a47c74625 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:08.257295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:08.262352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af921ff8-9064-4f12-86f5-cf3a47c74625 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:08.303212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a5173-5409-4b5e-9fff-d0384bd881b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:08.303912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5173-5409-4b5e-9fff-d0384bd881b2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:08.304186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5173-5409-4b5e-9fff-d0384bd881b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:08.304481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a5173-5409-4b5e-9fff-d0384bd881b2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:08.305125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a5173-5409-4b5e-9fff-d0384bd881b2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:08.306564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a5173-5409-4b5e-9fff-d0384bd881b2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-f69a5173-5409-4b5e-9fff-d0384bd881b2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:08.316485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:08.317583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:08.318414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 509/1010] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:47:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:08.323740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee0efe-cc53-4cfb-bfd7-c70013d2aa62 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:08.361066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:08.417165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03d02-d041-4772-badf-c4e66c8ebe01 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:08.417357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03d02-d041-4772-badf-c4e66c8ebe01 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:08.417620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03d02-d041-4772-badf-c4e66c8ebe01 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:08.419936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 510/1011] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:47:08.424863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9e7f-4b74-4a33-b4ce-2d03be2088d1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:08.425121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9e7f-4b74-4a33-b4ce-2d03be2088d1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:08.425393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c9e7f-4b74-4a33-b4ce-2d03be2088d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:08.431117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee0efe-cc53-4cfb-bfd7-c70013d2aa62 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:08.443669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-849c9e7f-4b74-4a33-b4ce-2d03be2088d1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:08.444846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-849c9e7f-4b74-4a33-b4ce-2d03be2088d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:08.467663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:08.468419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:08.468698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:08.469543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:08.470566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:08.470877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:08.472245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:08.477652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 502/1012] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:47:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:08.483130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:08.515486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 511/1013] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:08 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:08.580780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:08.584939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7773f4e-8e3c-4ef7-b12c-6e853794c0d1 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:08.618652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:08.619558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:08.620011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:08.620456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:08.621296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:08.621797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:08.623324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:08.628832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 503/1014] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Oct 3 15:47:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:08.637125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c9e7f-4b74-4a33-b4ce-2d03be2088d1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:08.731918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:08.739880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c9e7f-4b74-4a33-b4ce-2d03be2088d1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:08.775144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:08.775766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:08.776027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:08.776295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:08.777003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:08.778388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:08.789885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:08.808753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 512/1015] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:47:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:47:08.910814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ea0d0-e9a5-48b7-b610-7b5b95e361f4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:08.954906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b26100-5aeb-4ad7-bbc2-1bf39127e048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:08.955546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b26100-5aeb-4ad7-bbc2-1bf39127e048 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:08.955832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b26100-5aeb-4ad7-bbc2-1bf39127e048 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:08.956103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b26100-5aeb-4ad7-bbc2-1bf39127e048 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:08.957259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b26100-5aeb-4ad7-bbc2-1bf39127e048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:08.958709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b26100-5aeb-4ad7-bbc2-1bf39127e048 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:08.960323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 504/1016] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:08.966475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:09.077631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bce633-9546-45b1-b6e3-74bb6a62d44b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:09.115253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e5f18-11b3-4cae-8508-4be5df6a6aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:09.115865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e5f18-11b3-4cae-8508-4be5df6a6aa3 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:09.116146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e5f18-11b3-4cae-8508-4be5df6a6aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:09.116417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e5f18-11b3-4cae-8508-4be5df6a6aa3 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:09.117175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e5f18-11b3-4cae-8508-4be5df6a6aa3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:09.118654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e5f18-11b3-4cae-8508-4be5df6a6aa3 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:09.147184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 513/1017] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:47:08 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:47:09.285408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c17d9-a68b-480a-a453-bc266b277bef None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:09.285824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c17d9-a68b-480a-a453-bc266b277bef None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:09.286005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c17d9-a68b-480a-a453-bc266b277bef None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:09.287472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 505/1018] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:09.293153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309c2d1-4877-49a6-90a9-6acb1c07f681 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:09.293459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309c2d1-4877-49a6-90a9-6acb1c07f681 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:09.293703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309c2d1-4877-49a6-90a9-6acb1c07f681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:09.302962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:09.317617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-9309c2d1-4877-49a6-90a9-6acb1c07f681 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:09.319145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9309c2d1-4877-49a6-90a9-6acb1c07f681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:09.392326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 514/1019] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:09 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:09.658873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c17d9-a68b-480a-a453-bc266b277bef None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:09.689039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:09.766066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c17d9-a68b-480a-a453-bc266b277bef None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:09.801856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c39025-09fd-42f9-a1be-fe350ec83cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:09.802490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39025-09fd-42f9-a1be-fe350ec83cb2 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:09.802761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39025-09fd-42f9-a1be-fe350ec83cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:09.803026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c39025-09fd-42f9-a1be-fe350ec83cb2 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:09.803731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c39025-09fd-42f9-a1be-fe350ec83cb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:09.805117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c39025-09fd-42f9-a1be-fe350ec83cb2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:09.822110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 506/1020] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:47:09 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:47:11.078335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740300c-312a-4abc-9123-2b342fd87e8d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.078888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740300c-312a-4abc-9123-2b342fd87e8d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.079007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740300c-312a-4abc-9123-2b342fd87e8d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.080549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 515/1021] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:11.086055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee3a18-d990-4303-9a47-bcc9b9098cb6 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.086460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee3a18-d990-4303-9a47-bcc9b9098cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.086648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee3a18-d990-4303-9a47-bcc9b9098cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.107161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d3ee3a18-d990-4303-9a47-bcc9b9098cb6 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:11.108347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d3ee3a18-d990-4303-9a47-bcc9b9098cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:11.184764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 507/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:11.376325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12f809-0dd2-4858-8339-60113e5ec339 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.376529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12f809-0dd2-4858-8339-60113e5ec339 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.377019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12f809-0dd2-4858-8339-60113e5ec339 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.378536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 516/1023] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:11.384039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095a6c-a715-4b96-bfd4-15619e43e268 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.384329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095a6c-a715-4b96-bfd4-15619e43e268 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.384600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc095a6c-a715-4b96-bfd4-15619e43e268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.403532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bc095a6c-a715-4b96-bfd4-15619e43e268 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:11.404705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bc095a6c-a715-4b96-bfd4-15619e43e268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:11.413266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:11.454709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12f809-0dd2-4858-8339-60113e5ec339 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:11.477107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 508/1024] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:11.559997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12f809-0dd2-4858-8339-60113e5ec339 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:11.590990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:11.596445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:11.597490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.598223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.598716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.609180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:11.614830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:11.615743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:11.616600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:11.618149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 517/1025] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 15:47:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:11.623888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc095a6c-a715-4b96-bfd4-15619e43e268 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:11.726486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc095a6c-a715-4b96-bfd4-15619e43e268 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:11.758444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:11.761876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:11.762496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.763624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.763904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.764584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:11.764917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:11.766426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:11.775018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 509/1026] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 15:47:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:11.780687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:11.785134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:11.880751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb897f3-0fd4-469e-bc42-6d762a05f8e2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:11.886932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530bbda-f10e-4711-8457-169a5e9edeed None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:11.929206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:11.929886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.930122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.930433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.930988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:11.931701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:11.931810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:11.931899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:11.932164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:11.933193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:11.935269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 518/1027] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:11.940961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:11.945617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:11.947081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:11.961663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:11.962798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:11.963673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:11.963673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:11.963673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:11.963673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 510/1028] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:11.968329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.006704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.037994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.051132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bae31-7236-4801-afbc-d5bb18bbd983 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.081546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f05f6-f206-4c12-909a-7f4f83e5b1ed None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.085167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d21ed5-c306-4849-b828-dd6912f83ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.085778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d21ed5-c306-4849-b828-dd6912f83ef2 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.086056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d21ed5-c306-4849-b828-dd6912f83ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.086331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d21ed5-c306-4849-b828-dd6912f83ef2 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.086980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d21ed5-c306-4849-b828-dd6912f83ef2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.088434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d21ed5-c306-4849-b828-dd6912f83ef2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.111538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 519/1029] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:47:11 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:47:12.112632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e89160-3a6e-4c1d-afe9-a61d7b9fd78f None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.112938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e89160-3a6e-4c1d-afe9-a61d7b9fd78f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.113214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e89160-3a6e-4c1d-afe9-a61d7b9fd78f None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.114458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 520/1030] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:12.117097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.117784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.118074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.118341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.119042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:12.119351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.120290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae74ea-0970-43a4-9e05-ccbda16c9a49 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.120579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae74ea-0970-43a4-9e05-ccbda16c9a49 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.120832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.120947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae74ea-0970-43a4-9e05-ccbda16c9a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.140393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 511/1031] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:12.141829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.142728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cfae74ea-0970-43a4-9e05-ccbda16c9a49 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:12.144458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cfae74ea-0970-43a4-9e05-ccbda16c9a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:12.225013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 521/1032] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:12.265456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb222-fa30-4751-a689-41a73cc8e74c None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.284853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.305424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.306324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.306809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.307079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.318246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.319192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:47:12.331930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:12.333026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:12.333961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 512/1033] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:12.339553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae74ea-0970-43a4-9e05-ccbda16c9a49 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.394909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.443309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae74ea-0970-43a4-9e05-ccbda16c9a49 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.477191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.477793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.478066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.478330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.479054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:12.479349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.480791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.495531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.498093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 522/1034] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:12.504557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.601872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b44c20-722b-4db1-94e5-7838f117d943 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.612502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75caf7-a6d0-43b5-991b-5f2cd101570a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.615184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.637292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.637915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.638197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.638449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.639021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.640388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:12.649036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:12.650054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:12.653752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 513/1035] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:12.655970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.657079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.657803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.658081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.658349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.706512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.731448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.732942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.768964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc302986-d70f-4aee-80a2-59b7c6e63ac4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.805091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.805690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.805961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.806226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.806844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] RBAC: Adding query filter params (name=image) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:12.807140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.808489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.814050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 514/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:12.819492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.853314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 523/1037] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 15:47:12.854231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:12.857092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.931098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c58e6d-1c86-4875-9732-b509bda39428 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.933458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:12.961620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740dfe7-b9fc-4bf0-ad9b-3a0a72042498 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:12.971089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:12.971696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:12.971971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:12.972237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:12.973160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] RBAC: Adding query filter params (type=image) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:12.973450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:12.974811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:12.980288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 515/1038] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:12.987516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:13.003946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:13.004574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:13.004881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:13.005147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:13.017619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:13.019070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:13.033114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:13.034088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:13.035021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:13.035021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:13.035021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:13.035021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 524/1039] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:13.039675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:13.099255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a72b5-78f4-43ae-8e21-42c73e8d7e3d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:13.105174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:13.136271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:13.136930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:13.137176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:13.137446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:13.138140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] RBAC: Adding query filter params (interface=public, service_id=9645006aee8e43fda9bcc3b260e9a5d0, region_id=RegionOne) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:13.138446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:13.139807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:13.149593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 516/1040] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:47:12 2022] GET /identity/v3/endpoints?service_id=9645006aee8e43fda9bcc3b260e9a5d0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:13.150641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:13.153721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd68c-3ace-48ff-b3bf-d4bd7acc8918 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:13.186602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:13.191196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:13.191804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:13.192080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:13.192348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:13.193021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:13.193325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:13.194707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:13.214405 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 525/1041] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:13.220918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:13.263331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97b3cd-6cc8-493e-9e58-405ddcb204b2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:13.298956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:13.299558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:13.299829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:13.300091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:13.300687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:13.302138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:13.308947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 517/1042] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:47:13 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:47:13.339378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03039d-aa36-494c-8f56-63ab3f3f2840 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:13.359066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:13.381644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:13.382295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:13.382569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:13.382838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:13.390679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:13.392197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:47:13.404681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:13.405677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:13.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 526/1043] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:47:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:13.414038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:13.521838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:13.528260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7de4-6b2a-486d-b411-603dfcde1814 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:13.564042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d5d8f-d932-4371-9287-959e3000e586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:13.564646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d5d8f-d932-4371-9287-959e3000e586 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:13.564945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d5d8f-d932-4371-9287-959e3000e586 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:13.565215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d5d8f-d932-4371-9287-959e3000e586 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:13.565906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d5d8f-d932-4371-9287-959e3000e586 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:13.566208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d5d8f-d932-4371-9287-959e3000e586 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:13.567635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d5d8f-d932-4371-9287-959e3000e586 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:13.588207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 518/1044] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:47:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:13.589507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:13.692282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19110f56-5d44-4862-bf28-40b7f12e7194 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:13.725585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a30169-228f-43eb-8812-f5212cb662a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:13.726187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a30169-228f-43eb-8812-f5212cb662a9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:13.726462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a30169-228f-43eb-8812-f5212cb662a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:13.726729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a30169-228f-43eb-8812-f5212cb662a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:13.727449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a30169-228f-43eb-8812-f5212cb662a9 None admin] RBAC: Adding query filter params (role.id=f41f57df30b84ba5ae1b0472240664e7, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=ed959e4bdaf44132b05ccaec68c41172) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:13.727755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a30169-228f-43eb-8812-f5212cb662a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:13.729263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a30169-228f-43eb-8812-f5212cb662a9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:13.740773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 527/1045] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:13 2022] GET /identity/v3/role_assignments?user.id=ed959e4bdaf44132b05ccaec68c41172&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=f41f57df30b84ba5ae1b0472240664e7 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:14.407840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ceb87-75ad-434b-8d5d-232bae47f926 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:14.408350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ceb87-75ad-434b-8d5d-232bae47f926 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:14.408426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ceb87-75ad-434b-8d5d-232bae47f926 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:14.409779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 519/1046] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:14.415022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5cde-999f-43ea-805b-6585331c7d89 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:14.415419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5cde-999f-43ea-805b-6585331c7d89 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:14.415578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f5cde-999f-43ea-805b-6585331c7d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:14.434254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-400f5cde-999f-43ea-805b-6585331c7d89 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:14.435447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-400f5cde-999f-43ea-805b-6585331c7d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:14.470613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:14.506379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 528/1047] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:14.786518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ceb87-75ad-434b-8d5d-232bae47f926 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:14.890429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ceb87-75ad-434b-8d5d-232bae47f926 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:14.927112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:14.927756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:14.928036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:14.928307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:14.939305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:14.944088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:14.945229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:14.946075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:14.949056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 520/1048] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:47:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:14.951150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5cde-999f-43ea-805b-6585331c7d89 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:15.053130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f5cde-999f-43ea-805b-6585331c7d89 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:15.088556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:15.089190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.089503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.089772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.090440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:15.090743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:15.092123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:15.101151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 529/1049] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:15.106830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:15.126181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:15.208550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41981cff-c47f-4bd0-b0f1-46e3740e6881 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:15.243082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497d628-3752-4c03-a651-1103ad9f2141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:15.243849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497d628-3752-4c03-a651-1103ad9f2141 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.244229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497d628-3752-4c03-a651-1103ad9f2141 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.244600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497d628-3752-4c03-a651-1103ad9f2141 None admin] PATH_INFO: `/v3/users/e3a9a381343e49589d9c6d351384463d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.257882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497d628-3752-4c03-a651-1103ad9f2141 None admin] RBAC: Authorizing `identity:get_user(user_id=e3a9a381343e49589d9c6d351384463d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:15.259428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497d628-3752-4c03-a651-1103ad9f2141 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:15.261332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 521/1050] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:15 2022] GET /identity/v3/users/e3a9a381343e49589d9c6d351384463d => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:15.267462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:15.318260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:15.394447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6e5a4-5b75-4b02-b2fc-f641f7c654be None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:15.429023 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:15.429690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.429923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.430167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.483284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611640-8fe0-4ea4-936b-c22b93bf1e15 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.483847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611640-8fe0-4ea4-936b-c22b93bf1e15 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.483847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611640-8fe0-4ea4-936b-c22b93bf1e15 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.485441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 522/1051] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:47:15.490539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3d6e-755e-4932-84ac-672a5f2017a3 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.490815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3d6e-755e-4932-84ac-672a5f2017a3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.491084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3d6e-755e-4932-84ac-672a5f2017a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.506956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:15.508425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:15.509559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e5aa3d6e-755e-4932-84ac-672a5f2017a3 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:15.510803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e5aa3d6e-755e-4932-84ac-672a5f2017a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:15.540040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:15.579655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 523/1052] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:15.615977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 530/1053] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:15 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:47:15.621887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa3d6e-755e-4932-84ac-672a5f2017a3 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:15.626834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:15.725878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa3d6e-755e-4932-84ac-672a5f2017a3 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:15.759280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:15.759916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.760189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.760458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.761605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] RBAC: Authorizing `identity:get_project(project_id=1e7e930faf6c4a7ebcff4011d478612f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:15.763040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:15.770093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 524/1054] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:15 2022] GET /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:15.771224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:15.849605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:15.879410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef908176-8ce7-4498-be9a-14e33058006f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:15.913061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:15.918985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04989c08-f726-4c38-a0bf-64fdf2ac823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:15.919587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989c08-f726-4c38-a0bf-64fdf2ac823e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.919859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989c08-f726-4c38-a0bf-64fdf2ac823e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.920130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04989c08-f726-4c38-a0bf-64fdf2ac823e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.921057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04989c08-f726-4c38-a0bf-64fdf2ac823e None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=1e7e930faf6c4a7ebcff4011d478612f, user.id=e3a9a381343e49589d9c6d351384463d) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:15.921443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04989c08-f726-4c38-a0bf-64fdf2ac823e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:15.925008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04989c08-f726-4c38-a0bf-64fdf2ac823e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:15.929039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:15.942263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 531/1055] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:15 2022] GET /identity/v3/role_assignments?user.id=e3a9a381343e49589d9c6d351384463d&scope.project.id=1e7e930faf6c4a7ebcff4011d478612f&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:15.944946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8699e6-82ae-4568-baca-fc5c28e92bb5 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.945746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8699e6-82ae-4568-baca-fc5c28e92bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.945746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8699e6-82ae-4568-baca-fc5c28e92bb5 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.947167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 532/1056] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:47:15.953529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73666b1a-6aec-40e4-8b6d-fd89ed66481a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:15.953836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73666b1a-6aec-40e4-8b6d-fd89ed66481a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:15.954217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73666b1a-6aec-40e4-8b6d-fd89ed66481a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:15.974706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-73666b1a-6aec-40e4-8b6d-fd89ed66481a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:15.974948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0386b7-f21f-4bbd-ab8b-c0b4ab6d3104 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:15.976036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-73666b1a-6aec-40e4-8b6d-fd89ed66481a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:16.013997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:16.014758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:16.015036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:16.015313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] PATH_INFO: `/v3/services/image` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:16.016533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:16.018263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:16.028146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:16.029388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:16.030757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 525/1057] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Oct 3 15:47:15 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:16.036073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:16.059819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 533/1058] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:16.149908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330348a-a756-45c6-a497-dfda80523edf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:16.184817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:16.185429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:16.185707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:16.185976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:16.186598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] RBAC: Adding query filter params (name=image) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:16.186898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:16.188429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:16.190239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:16.194405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 526/1059] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:16.199929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73666b1a-6aec-40e4-8b6d-fd89ed66481a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:16.300369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73666b1a-6aec-40e4-8b6d-fd89ed66481a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:16.331935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:16.341627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:16.342243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:16.342356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:16.342526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:16.342783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:16.343415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] RBAC: Adding query filter params (type=image) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:16.344253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:16.345665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:16.352160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 534/1060] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:16.356342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:16.450309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aadf11-ecb8-4d73-ba08-52206cf9b892 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:16.466036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4159-5d05-434e-afb3-abbb0d806505 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:16.485967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:16.486730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:16.487006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:16.487271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:16.498439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:16.501655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:16.502261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:16.502541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:16.502810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:16.503201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:16.504089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:16.504969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:16.505619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:16.505695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 527/1061] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 15:47:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:16.509373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:16.510104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:16.541002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 535/1062] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:47:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:47:16.585826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:16.633854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767689c6-d84c-4888-a4da-347594b20a6e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:16.677365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:16.677975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:16.678254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:16.678529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:16.679203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:16.679507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:16.680936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:16.690605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 528/1063] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 15:47:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:16.696276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:16.745965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:16.803413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9147903-6fce-4341-a201-b8b6fd9ea834 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:16.837834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:16.838490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:16.838761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:16.839021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:16.851495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:16.852997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:16.868972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:16.870021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:16.870948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:16.870948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:16.870948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:16.870948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 536/1064] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:16.877507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:16.979168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03b1eb-6c19-4eb3-b8f1-50a1147e4c66 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:16.996319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:17.013953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:17.014563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:17.014840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:17.015107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:17.016004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:17.016305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:17.017701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:17.029541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 529/1065] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:17.035549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:17.136026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:17.140666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233a0eb-e35f-413f-9043-7aa7ec55232f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:17.179482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:17.180105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:17.180378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:17.180643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:17.188255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:17.189728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:47:17.200843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:17.201796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:17.202754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 537/1066] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:47:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:17.207662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:17.311989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fca0f8-341b-4d6c-808a-d7a348f17aa0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:17.349226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:17.349831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:17.350104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:17.350368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:17.351093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:17.351390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:17.352824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:17.370581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 530/1067] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:47:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:17.376661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:17.478579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11eae73-1dfb-4593-ad7c-ecb01b3c389a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:17.513227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:17.513850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:17.514125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:17.514392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:17.524858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:17.526302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:17.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:17.541483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:17.542839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:17.542839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:17.542839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:17.543246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 538/1068] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:17.547655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:17.582925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:17.649362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069199a5-0d0b-48bb-9422-5bd9e61618f0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:17.684163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:17.684798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:17.685081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:17.685350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:17.685990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:17.686292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:17.688273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:17.704874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 531/1069] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:17.706078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:17.719156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:17.810575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83984f-fbac-41d9-b8c8-870dcc4129fc None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:17.846885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:17.847436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:17.847965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:17.848234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:17.854958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:17.856427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:47:17.867661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:17.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:17.874235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 539/1070] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:17.875311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:17.975142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:17.982689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6fbd4-4f8d-429c-b3ae-fef68b457d4e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:18.017665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5bb42-f8fe-4f3e-ae50-188fb7b9b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:18.018272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5bb42-f8fe-4f3e-ae50-188fb7b9b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.018550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5bb42-f8fe-4f3e-ae50-188fb7b9b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.018827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5bb42-f8fe-4f3e-ae50-188fb7b9b4a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.019513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5bb42-f8fe-4f3e-ae50-188fb7b9b4a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:18.019816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5bb42-f8fe-4f3e-ae50-188fb7b9b4a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:18.021274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5bb42-f8fe-4f3e-ae50-188fb7b9b4a1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:18.038605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 532/1071] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:18.040195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:18.114248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:18.147948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39ec9-e130-40dc-a6f4-f3aa1deb44ea None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:18.152709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5f34f-c7e1-4d62-a267-1924d7320d53 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.153058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5f34f-c7e1-4d62-a267-1924d7320d53 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.153296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5f34f-c7e1-4d62-a267-1924d7320d53 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.154762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 533/1072] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:18.160283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0d5e3-8e35-4698-a78e-f4adf709e309 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.160545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0d5e3-8e35-4698-a78e-f4adf709e309 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.160843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0d5e3-8e35-4698-a78e-f4adf709e309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.180276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-04c0d5e3-8e35-4698-a78e-f4adf709e309 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:18.181506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-04c0d5e3-8e35-4698-a78e-f4adf709e309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:18.184131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:18.184999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.185278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.185552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/ed959e4bdaf44132b05ccaec68c41172/roles/f41f57df30b84ba5ae1b0472240664e7` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.187728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=ed959e4bdaf44132b05ccaec68c41172, role_id=f41f57df30b84ba5ae1b0472240664e7)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:18.189209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:18.216758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 540/1073] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:47:18 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/ed959e4bdaf44132b05ccaec68c41172/roles/f41f57df30b84ba5ae1b0472240664e7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:47:18.267979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 534/1074] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:18.538870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:18.643063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbebcf-fd6b-4e66-9086-f7388244f55e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:18.673993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:18.682547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c7e15-ab94-41e8-9916-fbcf3fbc02eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:18.683218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c7e15-ab94-41e8-9916-fbcf3fbc02eb None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.683649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c7e15-ab94-41e8-9916-fbcf3fbc02eb None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.683918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c7e15-ab94-41e8-9916-fbcf3fbc02eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.695510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c7e15-ab94-41e8-9916-fbcf3fbc02eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-926c7e15-ab94-41e8-9916-fbcf3fbc02eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:18.700192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:18.701351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:18.702197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:18.705069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 541/1075] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:18.707807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0d5e3-8e35-4698-a78e-f4adf709e309 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:18.748842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71e30c-66ed-4108-8cca-af1b3fd1c082 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.749129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71e30c-66ed-4108-8cca-af1b3fd1c082 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.749400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71e30c-66ed-4108-8cca-af1b3fd1c082 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.750988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 542/1076] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:18.755953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f38de-6f0a-4c0c-bf94-ee9845246f26 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.756232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f38de-6f0a-4c0c-bf94-ee9845246f26 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.756503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f38de-6f0a-4c0c-bf94-ee9845246f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.774753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-695f38de-6f0a-4c0c-bf94-ee9845246f26 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:18.775931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-695f38de-6f0a-4c0c-bf94-ee9845246f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:18.794065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:18.810573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0d5e3-8e35-4698-a78e-f4adf709e309 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:18.845398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 543/1077] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:18.845586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:18.846261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:18.846493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:18.846756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:18.847414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:18.847980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:18.849377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:18.862807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 535/1078] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:18.864602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f38de-6f0a-4c0c-bf94-ee9845246f26 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:18.969945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f38de-6f0a-4c0c-bf94-ee9845246f26 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:18.990050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:19.005399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:19.006026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:19.006299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:19.006565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] PATH_INFO: `/v3/users/e3a9a381343e49589d9c6d351384463d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:19.007690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] RBAC: Authorizing `identity:get_user(user_id=e3a9a381343e49589d9c6d351384463d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:19.009160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:19.015999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 544/1079] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:18 2022] GET /identity/v3/users/e3a9a381343e49589d9c6d351384463d => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:19.017600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:19.114984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:19.123070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa59f37-3249-4ecb-b654-94dfe57a8daf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:19.158637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:19.159262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:19.159535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:19.159801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:19.160941 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] RBAC: Authorizing `identity:get_project(project_id=1e7e930faf6c4a7ebcff4011d478612f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:19.162362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:19.164333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 536/1080] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:19 2022] GET /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:19.170025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:19.188905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:19.226640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d368bcf-82e0-47f4-9feb-22a31c5b9dac None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:19.262988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:19.263603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:19.263880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:19.264149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:19.277327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb4b63-8c20-4c0f-bfa8-d97af302d412 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:19.318548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:19.319663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:19.319663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:19.319818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f/users/e3a9a381343e49589d9c6d351384463d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:19.321814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e7e930faf6c4a7ebcff4011d478612f, user_id=e3a9a381343e49589d9c6d351384463d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:19.323244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:19.334940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:19.337605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 537/1081] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:47:19 2022] PUT /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f/users/e3a9a381343e49589d9c6d351384463d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:47:19.353798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:19.355744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:19.478343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 545/1082] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 15:47:19.484766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:19.570698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:19.596170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15509cd-f53a-40c5-858c-d4dfbb50e431 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:19.630549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8110170-dc75-4a2c-b31c-db50047bb024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:19.631174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110170-dc75-4a2c-b31c-db50047bb024 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:19.631451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110170-dc75-4a2c-b31c-db50047bb024 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:19.631721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8110170-dc75-4a2c-b31c-db50047bb024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:19.643069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8110170-dc75-4a2c-b31c-db50047bb024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:19.644573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8110170-dc75-4a2c-b31c-db50047bb024 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-b8110170-dc75-4a2c-b31c-db50047bb024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:19.658975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:19.659936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:19.660872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:19.660872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:19.660872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:19.660872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 538/1083] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:19.666043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:19.757835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:19.772359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37e691-9278-41b4-be7b-88933d556d5e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:19.808502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dbd7f-9b57-4771-aaaa-83d6fbcefa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:19.809145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dbd7f-9b57-4771-aaaa-83d6fbcefa2c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:19.809424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dbd7f-9b57-4771-aaaa-83d6fbcefa2c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:19.809696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dbd7f-9b57-4771-aaaa-83d6fbcefa2c None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:19.810374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dbd7f-9b57-4771-aaaa-83d6fbcefa2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:19.810682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dbd7f-9b57-4771-aaaa-83d6fbcefa2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:19.812080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dbd7f-9b57-4771-aaaa-83d6fbcefa2c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:19.825715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 546/1084] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:20.389024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa015e6a-ae38-485d-8375-4b16b6be117b None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:20.389554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa015e6a-ae38-485d-8375-4b16b6be117b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:20.389554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa015e6a-ae38-485d-8375-4b16b6be117b None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:20.390922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 539/1085] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:20.395999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ed4c-be39-4bc2-977c-7ac0c2add34e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:20.396294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ed4c-be39-4bc2-977c-7ac0c2add34e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:20.396571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ed4c-be39-4bc2-977c-7ac0c2add34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:20.414489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ae59ed4c-be39-4bc2-977c-7ac0c2add34e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:20.415676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ae59ed4c-be39-4bc2-977c-7ac0c2add34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:20.481523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 547/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:20.752513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa015e6a-ae38-485d-8375-4b16b6be117b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:20.853851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa015e6a-ae38-485d-8375-4b16b6be117b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:20.879605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:20.887760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:20.888380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:20.888660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:20.888956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:20.899699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:20.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:20.908762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:20.909899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 540/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Oct 3 15:47:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:20.911835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59ed4c-be39-4bc2-977c-7ac0c2add34e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:20.990482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:21.012326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59ed4c-be39-4bc2-977c-7ac0c2add34e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:21.046651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:21.047244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.047522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.047790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.048452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:21.048783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:21.050164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:21.057960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 548/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 15:47:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:21.063933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:21.161643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90653e8-e8da-44cb-85d6-4bb67c0e7db3 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:21.193931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:21.194546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.194817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.195081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.205251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:21.206683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:21.221101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:21.222181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:21.223346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:21.223346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:21.223346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:21.224872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 541/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:21.228058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:21.332581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5132c-ec3a-4794-b987-b641e5d01552 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:21.368029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c2baa-cf9a-4af7-8fb3-43a367f63d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:21.368633 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c2baa-cf9a-4af7-8fb3-43a367f63d09 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.368940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c2baa-cf9a-4af7-8fb3-43a367f63d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.369213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c2baa-cf9a-4af7-8fb3-43a367f63d09 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.369858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c2baa-cf9a-4af7-8fb3-43a367f63d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:21.370162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c2baa-cf9a-4af7-8fb3-43a367f63d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:21.371533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c2baa-cf9a-4af7-8fb3-43a367f63d09 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:21.382053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:21.390165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 549/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:21.391981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:21.436329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:21.495475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492621f-0d50-4e24-899c-a3415010f215 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:21.515632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82a84-cecc-476a-be86-61b26b361e51 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.515997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82a84-cecc-476a-be86-61b26b361e51 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.516349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a82a84-cecc-476a-be86-61b26b361e51 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.517803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 550/1091] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:47:21.522991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be34fe-8721-4ea3-8e31-00205e6c76a1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.523275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be34fe-8721-4ea3-8e31-00205e6c76a1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.523551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be34fe-8721-4ea3-8e31-00205e6c76a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.528734 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:21.529383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.529673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.529930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.538740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:21.540172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:21.541569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a4be34fe-8721-4ea3-8e31-00205e6c76a1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:21.542759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a4be34fe-8721-4ea3-8e31-00205e6c76a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:47:21.556932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:21.557981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:21.558943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 542/1092] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Oct 3 15:47:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:21.564270 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:21.618265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 551/1093] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:21.659549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:21.662997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edaef3-5411-4766-95b0-b2a389680526 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:21.696198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:21.696818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.697094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.697365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.698158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:21.698455 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:21.699864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:21.716071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 543/1094] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Oct 3 15:47:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:21.722968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be34fe-8721-4ea3-8e31-00205e6c76a1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:21.820135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be34fe-8721-4ea3-8e31-00205e6c76a1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:21.848519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:21.854505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:21.855097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:21.855370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:21.855634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:21.890581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:21.924566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:21.926080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:21.995529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb04b0c-0b55-4ade-a515-8f2c28bc3031 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.027964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 552/1095] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:47:22.030049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.030681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.031552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.031823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.034700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.042368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:22.047368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:22.048308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:22.049351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:22.052869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 544/1096] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:22.054240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.063602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.072654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.134141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600df7f-dc9c-4ef4-ac1b-7291de4b5303 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.155615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b06f4-df1b-434a-b030-565ac0160a44 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.168495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.169155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.169428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.169696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.179526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.180998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.189182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.189782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.190062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.190330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.190985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:22.191281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.192635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:22.196764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:22.197836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:47:22.198735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:22.198735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:47:22.198735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:22.198735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 553/1097] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:22.206504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 545/1098] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:22.206654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.207618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.285792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.286075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.304313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf8116-d727-4fde-87dd-331fe4d14ab2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.305801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa82ce8-5974-4cd3-8996-f62b92412cb6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.339441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.340046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.340319 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.340582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.340925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.341258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:22.341544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.341639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.341818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.342086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] PATH_INFO: `/v3/users/e3a9a381343e49589d9c6d351384463d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.342937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.343723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] RBAC: Authorizing `identity:get_user(user_id=e3a9a381343e49589d9c6d351384463d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.345171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.346730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 546/1099] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/users/e3a9a381343e49589d9c6d351384463d => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:22.352513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.361429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 554/1100] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:22.362527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.450156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72ba2-a46c-4dbd-9345-df59dbe8381b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.459497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972762c3-d11e-4472-be3e-775417b511c2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.483616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.484305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.484578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.484931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.493638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.494257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.494532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.494807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.501691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.501820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.503149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:47:22.514659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:22.516402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:22.517599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 555/1101] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:47:22.521961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.524532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.555772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.557257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.623573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67078248-0024-4f8f-a689-306174e28d3b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.657908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.658508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.658784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.659048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.659735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:22.660033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.661501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.661936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 547/1102] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 15:47:22.667727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.676991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 556/1103] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:22.682809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.716340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.743644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.769052 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6aa58-9c56-4d40-a08f-4472d3c5492b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.783013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a1f1-7a16-4ca2-bfba-9599ff556884 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.803512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.804161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.804437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.804707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.805835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] RBAC: Authorizing `identity:get_project(project_id=1e7e930faf6c4a7ebcff4011d478612f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.807403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.809033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 548/1104] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:22.815616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:22.817860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae84f5a-28a4-4ed4-a610-ec87a8f695ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.818458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae84f5a-28a4-4ed4-a610-ec87a8f695ef None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.818731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae84f5a-28a4-4ed4-a610-ec87a8f695ef None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.819003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae84f5a-28a4-4ed4-a610-ec87a8f695ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.819723 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae84f5a-28a4-4ed4-a610-ec87a8f695ef None admin] RBAC: Adding query filter params (role.id=f41f57df30b84ba5ae1b0472240664e7, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=ed959e4bdaf44132b05ccaec68c41172) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:22.820023 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae84f5a-28a4-4ed4-a610-ec87a8f695ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.821482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae84f5a-28a4-4ed4-a610-ec87a8f695ef None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.827614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 557/1105] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/role_assignments?user.id=ed959e4bdaf44132b05ccaec68c41172&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=f41f57df30b84ba5ae1b0472240664e7 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:22.919398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e2c20-9bb6-41e5-9740-7bf30371d0b1 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:22.937641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:22.956401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347df5e-62e7-499f-9d81-b22306bd1d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:22.957046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347df5e-62e7-499f-9d81-b22306bd1d07 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:22.957324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347df5e-62e7-499f-9d81-b22306bd1d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:22.957595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347df5e-62e7-499f-9d81-b22306bd1d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:22.958336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347df5e-62e7-499f-9d81-b22306bd1d07 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=1e7e930faf6c4a7ebcff4011d478612f, user.id=e3a9a381343e49589d9c6d351384463d) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:22.958644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347df5e-62e7-499f-9d81-b22306bd1d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:22.960091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347df5e-62e7-499f-9d81-b22306bd1d07 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:22.966665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 549/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:22 2022] GET /identity/v3/role_assignments?user.id=e3a9a381343e49589d9c6d351384463d&scope.project.id=1e7e930faf6c4a7ebcff4011d478612f&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:47:25.129925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22fbb2-71fb-4b54-823f-ad72adb2da3b None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:25.130462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22fbb2-71fb-4b54-823f-ad72adb2da3b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:25.130462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22fbb2-71fb-4b54-823f-ad72adb2da3b None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:25.131785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 558/1107] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:25.136911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d74935-98c9-4e73-a20d-545575b71ff8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:25.137343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d74935-98c9-4e73-a20d-545575b71ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:25.137462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d74935-98c9-4e73-a20d-545575b71ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:25.155803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-17d74935-98c9-4e73-a20d-545575b71ff8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:25.157098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-17d74935-98c9-4e73-a20d-545575b71ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:25.222937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 550/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:25.497670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22fbb2-71fb-4b54-823f-ad72adb2da3b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:25.600647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22fbb2-71fb-4b54-823f-ad72adb2da3b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:25.626713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:25.635413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc504d7-0871-47e4-bbb5-8a44ab8af568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:25.636025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc504d7-0871-47e4-bbb5-8a44ab8af568 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:25.636298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc504d7-0871-47e4-bbb5-8a44ab8af568 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:25.636564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc504d7-0871-47e4-bbb5-8a44ab8af568 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:25.637704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc504d7-0871-47e4-bbb5-8a44ab8af568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:25.639138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc504d7-0871-47e4-bbb5-8a44ab8af568 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:25.641757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 559/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:25.647391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d74935-98c9-4e73-a20d-545575b71ff8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:25.743020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d74935-98c9-4e73-a20d-545575b71ff8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:25.753348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:25.776937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c4387-5325-4e0a-9eb0-5b6931789575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:25.777539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c4387-5325-4e0a-9eb0-5b6931789575 None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:25.777819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c4387-5325-4e0a-9eb0-5b6931789575 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:25.778093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c4387-5325-4e0a-9eb0-5b6931789575 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:25.778750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c4387-5325-4e0a-9eb0-5b6931789575 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:25.780177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c4387-5325-4e0a-9eb0-5b6931789575 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:25.801384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 551/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:47:25 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:47:28.039937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a58b6-8821-4889-ab73-9aacad1717c2 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:28.040492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a58b6-8821-4889-ab73-9aacad1717c2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:28.040492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a58b6-8821-4889-ab73-9aacad1717c2 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:28.041820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 560/1111] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:28.047324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8f97-1d1d-4189-8238-97b265eb8017 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:28.047701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8f97-1d1d-4189-8238-97b265eb8017 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:28.047864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab8f97-1d1d-4189-8238-97b265eb8017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:28.066238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-1cab8f97-1d1d-4189-8238-97b265eb8017 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:28.067406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1cab8f97-1d1d-4189-8238-97b265eb8017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:28.133211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 552/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:28.400508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a58b6-8821-4889-ab73-9aacad1717c2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:28.500317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a58b6-8821-4889-ab73-9aacad1717c2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:28.533812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:28.534422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:28.534700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:28.534968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] PATH_INFO: `/v3/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:28.536364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] RBAC: Authorizing `identity:get_role(role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:28.537769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:28.539070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 561/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:28 2022] GET /identity/v3/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:28.544791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab8f97-1d1d-4189-8238-97b265eb8017 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:28.634994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:28.641267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab8f97-1d1d-4189-8238-97b265eb8017 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:28.674085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:28.674695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:28.674968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:28.675237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] PATH_INFO: `/v3/users/d6c9b63f672c4d18901374e9b27534ad` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:28.689048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] RBAC: Authorizing `identity:get_user(user_id=d6c9b63f672c4d18901374e9b27534ad)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:28.690478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:28.692025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 553/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:28 2022] GET /identity/v3/users/d6c9b63f672c4d18901374e9b27534ad => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:28.697701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:28.757560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:28.801456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee138e6c-da9a-4e49-8531-91374a8f7a2b None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:28.838258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:28.838855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:28.839136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:28.839406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:28.907233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:28.908621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:28.966972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:29.007455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 562/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:47:29.013406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:29.113596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c60e0-2482-4061-b5e7-636bf0808610 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:29.149662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689295bd-a7ab-4ab4-b419-ecce513fb0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:29.150287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689295bd-a7ab-4ab4-b419-ecce513fb0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:29.150564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689295bd-a7ab-4ab4-b419-ecce513fb0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:29.150834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689295bd-a7ab-4ab4-b419-ecce513fb0d7 None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:29.151952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689295bd-a7ab-4ab4-b419-ecce513fb0d7 None admin] RBAC: Authorizing `identity:get_project(project_id=1e7e930faf6c4a7ebcff4011d478612f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:29.153423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689295bd-a7ab-4ab4-b419-ecce513fb0d7 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:29.155043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 554/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:29 2022] GET /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:29.161819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:29.259025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea25105-626a-45d4-a9d0-38482e02a8bf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:29.292934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d31fb-eb0a-4efe-bac6-5751f860bf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:29.293521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d31fb-eb0a-4efe-bac6-5751f860bf08 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:29.293799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d31fb-eb0a-4efe-bac6-5751f860bf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:29.294074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d31fb-eb0a-4efe-bac6-5751f860bf08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:29.294796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d31fb-eb0a-4efe-bac6-5751f860bf08 None admin] RBAC: Adding query filter params (role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=1e7e930faf6c4a7ebcff4011d478612f, user.id=d6c9b63f672c4d18901374e9b27534ad) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:29.295106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d31fb-eb0a-4efe-bac6-5751f860bf08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:29.296632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d31fb-eb0a-4efe-bac6-5751f860bf08 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:29.302060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 563/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:29 2022] GET /identity/v3/role_assignments?user.id=d6c9b63f672c4d18901374e9b27534ad&scope.project.id=1e7e930faf6c4a7ebcff4011d478612f&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:31.549456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4475c3-e0ba-4a3a-a63f-71474104625c None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:31.550012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4475c3-e0ba-4a3a-a63f-71474104625c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:31.550012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4475c3-e0ba-4a3a-a63f-71474104625c None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:31.551325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 555/1118] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:31.556377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efcde3-1b1f-43aa-9432-0c5927f9b6c7 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:31.556771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efcde3-1b1f-43aa-9432-0c5927f9b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:31.556948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efcde3-1b1f-43aa-9432-0c5927f9b6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:31.575398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-72efcde3-1b1f-43aa-9432-0c5927f9b6c7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:31.576497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-72efcde3-1b1f-43aa-9432-0c5927f9b6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:31.596012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:31.641494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 564/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:31.908986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4475c3-e0ba-4a3a-a63f-71474104625c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:31.933214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:32.008375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4475c3-e0ba-4a3a-a63f-71474104625c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:32.040904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:32.041515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:32.041788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:32.042055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] PATH_INFO: `/v3/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:32.043422 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] RBAC: Authorizing `identity:get_role(role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:32.044795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:32.046673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 556/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:31 2022] GET /identity/v3/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:32.052586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efcde3-1b1f-43aa-9432-0c5927f9b6c7 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:32.147655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efcde3-1b1f-43aa-9432-0c5927f9b6c7 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:32.181454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:32.182004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:32.182277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:32.182541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] PATH_INFO: `/v3/users/d6c9b63f672c4d18901374e9b27534ad` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:32.183634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] RBAC: Authorizing `identity:get_user(user_id=d6c9b63f672c4d18901374e9b27534ad)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:32.185084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:32.186941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 565/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:32 2022] GET /identity/v3/users/d6c9b63f672c4d18901374e9b27534ad => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:32.192825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:32.279263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:32.289224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9a58b-ba69-4da1-b012-cd0ef2ce9eac None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:32.321515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b436-0939-4cff-bc40-82e0645d33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:32.322136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b436-0939-4cff-bc40-82e0645d33af None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:32.322413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b436-0939-4cff-bc40-82e0645d33af None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:32.322683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b436-0939-4cff-bc40-82e0645d33af None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:32.323789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0b436-0939-4cff-bc40-82e0645d33af None admin] RBAC: Authorizing `identity:get_project(project_id=1e7e930faf6c4a7ebcff4011d478612f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:32.325253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0b436-0939-4cff-bc40-82e0645d33af None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:32.326861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 557/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:32 2022] GET /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:32.333362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:32.359065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:32.430990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ad7fb-d37e-420b-8436-c8c70e3c3754 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:32.463104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900960e-fbf3-4a5b-a05e-ae9de86d2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:32.463676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900960e-fbf3-4a5b-a05e-ae9de86d2499 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:32.463946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900960e-fbf3-4a5b-a05e-ae9de86d2499 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:32.464210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900960e-fbf3-4a5b-a05e-ae9de86d2499 None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f/users/d6c9b63f672c4d18901374e9b27534ad/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:32.466379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900960e-fbf3-4a5b-a05e-ae9de86d2499 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e7e930faf6c4a7ebcff4011d478612f, user_id=d6c9b63f672c4d18901374e9b27534ad, role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:32.468370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900960e-fbf3-4a5b-a05e-ae9de86d2499 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:32.481323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 566/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:47:32 2022] PUT /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f/users/d6c9b63f672c4d18901374e9b27534ad/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:47:34.650860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58a757-0b70-4c9f-88a5-3efc94be0866 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:34.651375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58a757-0b70-4c9f-88a5-3efc94be0866 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:34.651375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58a757-0b70-4c9f-88a5-3efc94be0866 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:34.652622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 558/1124] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:34.658050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab73d4d-5981-4d39-968b-a5f4a0c60dcb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:34.658500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab73d4d-5981-4d39-968b-a5f4a0c60dcb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:34.658605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab73d4d-5981-4d39-968b-a5f4a0c60dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:34.677242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-bab73d4d-5981-4d39-968b-a5f4a0c60dcb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:34.678400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bab73d4d-5981-4d39-968b-a5f4a0c60dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:34.742912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 567/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:35.007563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58a757-0b70-4c9f-88a5-3efc94be0866 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:35.105758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58a757-0b70-4c9f-88a5-3efc94be0866 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:35.137930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:35.138531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:35.138804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:35.139069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] PATH_INFO: `/v3/roles/e976a4b575914c8d9a0a5796a47d34bb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:35.141151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] RBAC: Authorizing `identity:get_role(role_id=e976a4b575914c8d9a0a5796a47d34bb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:35.142509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:35.143793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 559/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:35 2022] GET /identity/v3/roles/e976a4b575914c8d9a0a5796a47d34bb => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:35.149514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab73d4d-5981-4d39-968b-a5f4a0c60dcb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:35.164108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:35.246799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab73d4d-5981-4d39-968b-a5f4a0c60dcb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:35.279626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:35.280199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:35.280474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:35.280761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] PATH_INFO: `/v3/users/d6c9b63f672c4d18901374e9b27534ad` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:35.281860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=d6c9b63f672c4d18901374e9b27534ad)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:35.283291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:35.284924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 568/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:35 2022] GET /identity/v3/users/d6c9b63f672c4d18901374e9b27534ad => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:35.290704 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:35.318796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:35.388346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7023-1ce0-412a-ae13-84aa94f952ab None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:35.419711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:35.420302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:35.420580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:35.421392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:35.486126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:35.487567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:35.512312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:35.581699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 560/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:47:35.587621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:35.667621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:35.685950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3db6c-26a9-49b1-a3f9-407eb6d54aa5 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:35.718253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9fdff-7ad4-472a-9786-2dabd10f8d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:35.718936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9fdff-7ad4-472a-9786-2dabd10f8d2a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:35.719209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9fdff-7ad4-472a-9786-2dabd10f8d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:35.719475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9fdff-7ad4-472a-9786-2dabd10f8d2a None admin] PATH_INFO: `/v3/projects/1e7e930faf6c4a7ebcff4011d478612f` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:35.720572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9fdff-7ad4-472a-9786-2dabd10f8d2a None admin] RBAC: Authorizing `identity:get_project(project_id=1e7e930faf6c4a7ebcff4011d478612f)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:35.722120 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9fdff-7ad4-472a-9786-2dabd10f8d2a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:35.723973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 569/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:35 2022] GET /identity/v3/projects/1e7e930faf6c4a7ebcff4011d478612f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:35.730098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:35.827603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdb23a-8c53-418b-84d5-3ee9cdb14608 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:35.852661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:35.862188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2d928-7e6a-4e59-8d76-3ddbf71ab273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:35.862810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2d928-7e6a-4e59-8d76-3ddbf71ab273 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:35.863089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2d928-7e6a-4e59-8d76-3ddbf71ab273 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:35.863355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2d928-7e6a-4e59-8d76-3ddbf71ab273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:35.864068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2d928-7e6a-4e59-8d76-3ddbf71ab273 None admin] RBAC: Adding query filter params (role.id=e976a4b575914c8d9a0a5796a47d34bb, scope.project.id=1e7e930faf6c4a7ebcff4011d478612f, user.id=d6c9b63f672c4d18901374e9b27534ad) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:35.864366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2d928-7e6a-4e59-8d76-3ddbf71ab273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:35.865810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2d928-7e6a-4e59-8d76-3ddbf71ab273 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:35.871344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 561/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:35 2022] GET /identity/v3/role_assignments?user.id=d6c9b63f672c4d18901374e9b27534ad&scope.project.id=1e7e930faf6c4a7ebcff4011d478612f&role.id=e976a4b575914c8d9a0a5796a47d34bb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:37.999363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1138cf-62de-4e3d-a58b-c38399e1c396 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:37.999898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1138cf-62de-4e3d-a58b-c38399e1c396 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:37.999898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1138cf-62de-4e3d-a58b-c38399e1c396 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:38.001287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 570/1131] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:38.006526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2cc1-4a64-4c09-a27b-e490826983df None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:38.006919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2cc1-4a64-4c09-a27b-e490826983df None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:38.007063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2cc1-4a64-4c09-a27b-e490826983df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:38.025017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c91d2cc1-4a64-4c09-a27b-e490826983df None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:38.026178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c91d2cc1-4a64-4c09-a27b-e490826983df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:38.089657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 562/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:38.367787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1138cf-62de-4e3d-a58b-c38399e1c396 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:38.466943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1138cf-62de-4e3d-a58b-c38399e1c396 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:38.502646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f4f16-8faa-418c-acae-efef5690e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:38.503454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4f16-8faa-418c-acae-efef5690e538 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:38.503731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4f16-8faa-418c-acae-efef5690e538 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:38.503995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4f16-8faa-418c-acae-efef5690e538 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:38.505134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f4f16-8faa-418c-acae-efef5690e538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:38.506826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f4f16-8faa-418c-acae-efef5690e538 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:38.508395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 571/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:38.514756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d2cc1-4a64-4c09-a27b-e490826983df None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:38.611708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d2cc1-4a64-4c09-a27b-e490826983df None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:38.631079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:38.646073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391b1b4-4bd9-4c4f-8e72-23d5fc68c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:38.646673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391b1b4-4bd9-4c4f-8e72-23d5fc68c06c None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:38.646950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391b1b4-4bd9-4c4f-8e72-23d5fc68c06c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:38.647218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391b1b4-4bd9-4c4f-8e72-23d5fc68c06c None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:38.647860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391b1b4-4bd9-4c4f-8e72-23d5fc68c06c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:38.649320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391b1b4-4bd9-4c4f-8e72-23d5fc68c06c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:38.675878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 563/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:47:38 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:47:40.878026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ca0b6-f81a-4eca-9d37-c716c6170599 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:40.878549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ca0b6-f81a-4eca-9d37-c716c6170599 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:40.878549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ca0b6-f81a-4eca-9d37-c716c6170599 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:40.879891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 572/1135] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:40.885389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95f254-b8d7-49c0-ae7e-f3f9fe088c3b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:40.885765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95f254-b8d7-49c0-ae7e-f3f9fe088c3b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:40.885932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95f254-b8d7-49c0-ae7e-f3f9fe088c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:40.903849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0c95f254-b8d7-49c0-ae7e-f3f9fe088c3b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:40.905694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0c95f254-b8d7-49c0-ae7e-f3f9fe088c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:40.970091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 564/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:41.241336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ca0b6-f81a-4eca-9d37-c716c6170599 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:41.260497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:41.347420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ca0b6-f81a-4eca-9d37-c716c6170599 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:41.380732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fd788-3e69-4391-ba2e-77e3e7f8e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:41.381358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fd788-3e69-4391-ba2e-77e3e7f8e98a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:41.381632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fd788-3e69-4391-ba2e-77e3e7f8e98a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:41.381897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fd788-3e69-4391-ba2e-77e3e7f8e98a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:41.382994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fd788-3e69-4391-ba2e-77e3e7f8e98a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:41.384419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fd788-3e69-4391-ba2e-77e3e7f8e98a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:41.386008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 573/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:47:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:41.392005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95f254-b8d7-49c0-ae7e-f3f9fe088c3b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:41.486646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95f254-b8d7-49c0-ae7e-f3f9fe088c3b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:41.488107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:41.522480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c44a88-0d7a-4c2c-91ba-b4dc0f13a00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:41.523012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c44a88-0d7a-4c2c-91ba-b4dc0f13a00d None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:41.523287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c44a88-0d7a-4c2c-91ba-b4dc0f13a00d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:41.523557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c44a88-0d7a-4c2c-91ba-b4dc0f13a00d None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:41.524212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c44a88-0d7a-4c2c-91ba-b4dc0f13a00d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:41.525658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c44a88-0d7a-4c2c-91ba-b4dc0f13a00d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:41.546097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 565/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:47:41 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:47:43.703983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d404c-749d-4b45-b034-38c4e9ed5c91 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:43.704485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d404c-749d-4b45-b034-38c4e9ed5c91 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:43.704485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d404c-749d-4b45-b034-38c4e9ed5c91 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:43.705832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 574/1139] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:43.710974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafafd-38dc-487a-b47d-c859aede11fe None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:43.711397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafafd-38dc-487a-b47d-c859aede11fe None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:43.711507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafafd-38dc-487a-b47d-c859aede11fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:43.730006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ccbafafd-38dc-487a-b47d-c859aede11fe None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:43.731170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ccbafafd-38dc-487a-b47d-c859aede11fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:43.794314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 566/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:47:44.059452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d404c-749d-4b45-b034-38c4e9ed5c91 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:44.147222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:44.164120 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d404c-749d-4b45-b034-38c4e9ed5c91 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:44.199543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:44.200160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:44.200436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:44.200702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:44.211180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:44.215903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:44.217014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:44.218170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 575/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:47:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:44.223382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafafd-38dc-487a-b47d-c859aede11fe None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:44.308369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:44.321334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafafd-38dc-487a-b47d-c859aede11fe None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:44.354013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:44.354607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:44.354886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:44.355155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:44.355820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:44.356121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:44.357517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:44.365754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 567/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:47:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:44.371647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:44.469827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810748d7-8f96-4005-9a68-882f51f82575 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:44.503643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:44.504256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:44.504533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:44.504828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] PATH_INFO: `/v3/users/d5f5499c84d9422084db6b429b8f9998` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:44.517967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=d5f5499c84d9422084db6b429b8f9998)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:44.519423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:44.521085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 576/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:44 2022] GET /identity/v3/users/d5f5499c84d9422084db6b429b8f9998 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:44.526921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:44.623358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8d40f-c393-4d18-99f5-f63b685f475a None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:44.655730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:44.656315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:44.656587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:44.656882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:44.674874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:44.721199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:44.722892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:44.815900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 568/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:47:44.822041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:44.827330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:44.923759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98113a-b4b4-47b0-9165-35f1ca293dd8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:44.956164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00180429-ad7a-4d31-a1f7-005c6449bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:44.956815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00180429-ad7a-4d31-a1f7-005c6449bf10 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:44.957093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00180429-ad7a-4d31-a1f7-005c6449bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:44.957363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00180429-ad7a-4d31-a1f7-005c6449bf10 None admin] PATH_INFO: `/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:44.958485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00180429-ad7a-4d31-a1f7-005c6449bf10 None admin] RBAC: Authorizing `identity:get_project(project_id=0f96468c49fd4459b603dcc8d5b6baaa)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:44.959930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00180429-ad7a-4d31-a1f7-005c6449bf10 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:44.961597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 577/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:44 2022] GET /identity/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:44.968336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:45.017701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:45.065895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6785cb-0b2a-449a-8939-285267ca3958 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:45.099881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466708-ae70-4200-aa84-fdf29cc01b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:45.100468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466708-ae70-4200-aa84-fdf29cc01b82 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:45.100765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466708-ae70-4200-aa84-fdf29cc01b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:45.101039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466708-ae70-4200-aa84-fdf29cc01b82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:45.101759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e466708-ae70-4200-aa84-fdf29cc01b82 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=0f96468c49fd4459b603dcc8d5b6baaa, user.id=d5f5499c84d9422084db6b429b8f9998) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:45.102062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e466708-ae70-4200-aa84-fdf29cc01b82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:45.103480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e466708-ae70-4200-aa84-fdf29cc01b82 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:45.109434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 569/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:44 2022] GET /identity/v3/role_assignments?user.id=d5f5499c84d9422084db6b429b8f9998&scope.project.id=0f96468c49fd4459b603dcc8d5b6baaa&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:47.209618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf77c27-2419-42cf-a007-2fc5fa0edb6c None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:47.210193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf77c27-2419-42cf-a007-2fc5fa0edb6c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:47.210193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf77c27-2419-42cf-a007-2fc5fa0edb6c None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:47.211477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 578/1147] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:47.216748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301331-fe4e-4479-8885-10b30fb0a0c1 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:47.217869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301331-fe4e-4479-8885-10b30fb0a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:47.217869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301331-fe4e-4479-8885-10b30fb0a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:47.237581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b1301331-fe4e-4479-8885-10b30fb0a0c1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:47.238755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b1301331-fe4e-4479-8885-10b30fb0a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:47.309149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 570/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:47.579243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf77c27-2419-42cf-a007-2fc5fa0edb6c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:47.649947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:47.679635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf77c27-2419-42cf-a007-2fc5fa0edb6c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:47.713669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:47.714281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:47.714571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:47.714822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:47.725171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:47.729820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:47.730964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:47.732037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 579/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:47:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:47.736908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1301331-fe4e-4479-8885-10b30fb0a0c1 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:47.781731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:47.835776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1301331-fe4e-4479-8885-10b30fb0a0c1 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:47.868357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:47.868978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:47.869258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:47.869529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:47.870188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:47.870489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:47.871848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:47.880130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 571/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:47:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:47.885863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:47.986126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537e0d-525a-42ad-9b8d-ee7a9edd4602 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:48.015293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:48.020444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:48.021086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:48.021365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:48.021634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] PATH_INFO: `/v3/users/d5f5499c84d9422084db6b429b8f9998` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:48.022753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] RBAC: Authorizing `identity:get_user(user_id=d5f5499c84d9422084db6b429b8f9998)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:48.024197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:48.025781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 580/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:47 2022] GET /identity/v3/users/d5f5499c84d9422084db6b429b8f9998 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:48.031559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:48.129725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d1486-a813-443a-a360-07e608ce6c35 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:48.136401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:48.162573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c6eeb-8fe9-4358-95fb-278751b7dd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:48.163201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c6eeb-8fe9-4358-95fb-278751b7dd87 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:48.163477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c6eeb-8fe9-4358-95fb-278751b7dd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:48.163747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c6eeb-8fe9-4358-95fb-278751b7dd87 None admin] PATH_INFO: `/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:48.164886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c6eeb-8fe9-4358-95fb-278751b7dd87 None admin] RBAC: Authorizing `identity:get_project(project_id=0f96468c49fd4459b603dcc8d5b6baaa)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:48.166318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c6eeb-8fe9-4358-95fb-278751b7dd87 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:48.167926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 572/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:48 2022] GET /identity/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:48.174576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:48.272612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97d85-70a4-4729-822b-f73db90bfaa7 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:48.305415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81ddb0-de1c-4dab-8bc1-7dd72b22614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:48.306091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81ddb0-de1c-4dab-8bc1-7dd72b22614b None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:48.306364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81ddb0-de1c-4dab-8bc1-7dd72b22614b None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:48.306631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81ddb0-de1c-4dab-8bc1-7dd72b22614b None admin] PATH_INFO: `/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa/users/d5f5499c84d9422084db6b429b8f9998/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:48.308494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81ddb0-de1c-4dab-8bc1-7dd72b22614b None admin] RBAC: Authorizing `identity:create_grant(project_id=0f96468c49fd4459b603dcc8d5b6baaa, user_id=d5f5499c84d9422084db6b429b8f9998, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:48.310018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81ddb0-de1c-4dab-8bc1-7dd72b22614b None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:48.324474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 581/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:47:48 2022] PUT /identity/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa/users/d5f5499c84d9422084db6b429b8f9998/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:47:50.452254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a5a1-3d83-4539-bf3d-a94eb738aa41 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:50.452835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a5a1-3d83-4539-bf3d-a94eb738aa41 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:50.452835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a5a1-3d83-4539-bf3d-a94eb738aa41 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:50.454130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 573/1154] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:50.459306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772a7de-595f-47e9-a8ef-c81c5cceb0df None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:50.459690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772a7de-595f-47e9-a8ef-c81c5cceb0df None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:50.459851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772a7de-595f-47e9-a8ef-c81c5cceb0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:50.478330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c772a7de-595f-47e9-a8ef-c81c5cceb0df None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:50.479523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c772a7de-595f-47e9-a8ef-c81c5cceb0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:50.533960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:50.546313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 582/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:50.820223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355a5a1-3d83-4539-bf3d-a94eb738aa41 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:50.919713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355a5a1-3d83-4539-bf3d-a94eb738aa41 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:50.954781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:50.955403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:50.955682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:50.955957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:50.965891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:47:50.971414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:50.972804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:47:50.973871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 574/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:47:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:50.978848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772a7de-595f-47e9-a8ef-c81c5cceb0df None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:51.077358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772a7de-595f-47e9-a8ef-c81c5cceb0df None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:51.110935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:51.111530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:51.111811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:51.112084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:51.112767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:51.113073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:51.114454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:51.122353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 583/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:51.128445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:51.163385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:51.228211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34c7ad-fd3d-498c-bbec-692e8ad893fa None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:51.260760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:51.261376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:51.261654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:51.261926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] PATH_INFO: `/v3/users/d5f5499c84d9422084db6b429b8f9998` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:51.263051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] RBAC: Authorizing `identity:get_user(user_id=d5f5499c84d9422084db6b429b8f9998)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:51.265122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:51.266687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 575/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:47:51 2022] GET /identity/v3/users/d5f5499c84d9422084db6b429b8f9998 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:51.272434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:51.327980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:51.374355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06593caf-79dc-421a-a1e5-f2ad5494338f None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:51.407454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:51.408058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:51.408334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:51.408606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:51.476976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:51.478428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:51.529586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:51.578266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 584/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:47:51.584292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:51.691022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee26e0f-0918-45ec-b8f2-16024f6eb1bf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:51.692577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:51.725658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7001c-978b-48f5-aa17-2a5b2eb3f20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:51.726276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7001c-978b-48f5-aa17-2a5b2eb3f20a None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:51.726552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7001c-978b-48f5-aa17-2a5b2eb3f20a None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:51.726822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7001c-978b-48f5-aa17-2a5b2eb3f20a None admin] PATH_INFO: `/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:51.727923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7001c-978b-48f5-aa17-2a5b2eb3f20a None admin] RBAC: Authorizing `identity:get_project(project_id=0f96468c49fd4459b603dcc8d5b6baaa)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:51.729451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7001c-978b-48f5-aa17-2a5b2eb3f20a None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:51.731062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 576/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:47:51 2022] GET /identity/v3/projects/0f96468c49fd4459b603dcc8d5b6baaa => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:51.738156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:51.836448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac58d-a44e-4881-9b29-65cfb45e0d3c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:51.871061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130aaa5-b340-4bde-b9ff-76e84cc25e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:51.871649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130aaa5-b340-4bde-b9ff-76e84cc25e65 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:51.871920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130aaa5-b340-4bde-b9ff-76e84cc25e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:51.872186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130aaa5-b340-4bde-b9ff-76e84cc25e65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:51.873719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130aaa5-b340-4bde-b9ff-76e84cc25e65 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=0f96468c49fd4459b603dcc8d5b6baaa, user.id=d5f5499c84d9422084db6b429b8f9998) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:51.874409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130aaa5-b340-4bde-b9ff-76e84cc25e65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:51.875850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130aaa5-b340-4bde-b9ff-76e84cc25e65 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:51.881561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 585/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:47:51 2022] GET /identity/v3/role_assignments?user.id=d5f5499c84d9422084db6b429b8f9998&scope.project.id=0f96468c49fd4459b603dcc8d5b6baaa&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:54.057086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f3646-6cdc-45d6-8e41-988bbde89bdd None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:54.057619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f3646-6cdc-45d6-8e41-988bbde89bdd None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:54.057619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f3646-6cdc-45d6-8e41-988bbde89bdd None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:54.058943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 577/1162] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:54.064195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efa96-5366-447e-a3da-aafd0d9b3984 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:54.064605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efa96-5366-447e-a3da-aafd0d9b3984 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:54.064762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553efa96-5366-447e-a3da-aafd0d9b3984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:54.082909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-553efa96-5366-447e-a3da-aafd0d9b3984 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:54.084075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-553efa96-5366-447e-a3da-aafd0d9b3984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:54.098060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:54.156977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 586/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:54.428529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f3646-6cdc-45d6-8e41-988bbde89bdd None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:54.531359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f3646-6cdc-45d6-8e41-988bbde89bdd None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:54.564814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:54.565405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:54.565678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:54.565943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:54.603779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:54.633183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:54.634621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:54.734381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 578/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:47:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:47:54.740374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553efa96-5366-447e-a3da-aafd0d9b3984 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:54.835446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553efa96-5366-447e-a3da-aafd0d9b3984 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:54.868622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42ae58-2a21-49ec-aeba-d8abce8a9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:54.869198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42ae58-2a21-49ec-aeba-d8abce8a9808 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:54.869470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42ae58-2a21-49ec-aeba-d8abce8a9808 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:54.869736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42ae58-2a21-49ec-aeba-d8abce8a9808 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:54.876421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42ae58-2a21-49ec-aeba-d8abce8a9808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:54.877891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec42ae58-2a21-49ec-aeba-d8abce8a9808 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-ec42ae58-2a21-49ec-aeba-d8abce8a9808 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:47:54.889812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:47:54.891101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:47:54.892069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 587/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:47:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:47:54.898318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:54.971144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:55.004787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a757eb-7356-4fe2-9ec0-c147fc1f8e01 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:55.038602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e281c7b-d670-4f9a-a876-768ba4417916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:55.039186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281c7b-d670-4f9a-a876-768ba4417916 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:55.039457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281c7b-d670-4f9a-a876-768ba4417916 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:55.039720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e281c7b-d670-4f9a-a876-768ba4417916 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:55.040349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e281c7b-d670-4f9a-a876-768ba4417916 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:47:55.040645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e281c7b-d670-4f9a-a876-768ba4417916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:55.042031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e281c7b-d670-4f9a-a876-768ba4417916 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:55.049074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 579/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Oct 3 15:47:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:47:57.200197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176d08-d39d-4b57-842b-eff8f13e2a17 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:57.200764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176d08-d39d-4b57-842b-eff8f13e2a17 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:57.200764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176d08-d39d-4b57-842b-eff8f13e2a17 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:57.202272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 588/1167] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:57.207423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7f53b-c35b-49b2-8984-2c67f33451a1 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:57.207846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7f53b-c35b-49b2-8984-2c67f33451a1 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:57.207978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7f53b-c35b-49b2-8984-2c67f33451a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:57.226507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d3a7f53b-c35b-49b2-8984-2c67f33451a1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:57.227689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d3a7f53b-c35b-49b2-8984-2c67f33451a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:47:57.292388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 580/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:47:57.566527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176d08-d39d-4b57-842b-eff8f13e2a17 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:47:57.575494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:47:57.668939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176d08-d39d-4b57-842b-eff8f13e2a17 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:47:57.702683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4b380-91cb-4565-8dd1-74aa0b422d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:47:57.703273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4b380-91cb-4565-8dd1-74aa0b422d1d None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:57.703548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4b380-91cb-4565-8dd1-74aa0b422d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:57.703815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4b380-91cb-4565-8dd1-74aa0b422d1d None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:57.704476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4b380-91cb-4565-8dd1-74aa0b422d1d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:47:57.705879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4b380-91cb-4565-8dd1-74aa0b422d1d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:47:57.721648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 589/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Mon Oct 3 15:47:57 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:47:59.963829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8249cca-3e23-4b94-935d-a6170ba2f0ba None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:59.964395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8249cca-3e23-4b94-935d-a6170ba2f0ba None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:59.964395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8249cca-3e23-4b94-935d-a6170ba2f0ba None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:59.965730 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 581/1170] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:47:59.971523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36951ae2-9443-44e0-9d65-6bab9ff3cb66 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:47:59.972209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36951ae2-9443-44e0-9d65-6bab9ff3cb66 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:47:59.972313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36951ae2-9443-44e0-9d65-6bab9ff3cb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:47:59.990544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-36951ae2-9443-44e0-9d65-6bab9ff3cb66 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:47:59.991702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-36951ae2-9443-44e0-9d65-6bab9ff3cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:48:00.049492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:00.057845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 590/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:48:00.337133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8249cca-3e23-4b94-935d-a6170ba2f0ba None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:00.399923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:00.436674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8249cca-3e23-4b94-935d-a6170ba2f0ba None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:00.468850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b918cb-e8c8-4af7-adb5-1703a8af91ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:00.469461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b918cb-e8c8-4af7-adb5-1703a8af91ac None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:00.469735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b918cb-e8c8-4af7-adb5-1703a8af91ac None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:00.470002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b918cb-e8c8-4af7-adb5-1703a8af91ac None admin] PATH_INFO: `/v3/domains/baf5ee03ad0f469db6e177d5480d5950` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:00.480087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b918cb-e8c8-4af7-adb5-1703a8af91ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=baf5ee03ad0f469db6e177d5480d5950)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:00.481531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b918cb-e8c8-4af7-adb5-1703a8af91ac None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:00.483261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 582/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Oct 3 15:48:00 2022] GET /identity/v3/domains/baf5ee03ad0f469db6e177d5480d5950 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:00.489125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36951ae2-9443-44e0-9d65-6bab9ff3cb66 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:00.584787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36951ae2-9443-44e0-9d65-6bab9ff3cb66 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:00.619126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2ec6b-b65b-46fa-8466-9e49aba39a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:00.619712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2ec6b-b65b-46fa-8466-9e49aba39a86 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:00.619983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2ec6b-b65b-46fa-8466-9e49aba39a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:00.620250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2ec6b-b65b-46fa-8466-9e49aba39a86 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:00.620923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2ec6b-b65b-46fa-8466-9e49aba39a86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:00.622354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2ec6b-b65b-46fa-8466-9e49aba39a86 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:00.651041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 591/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:48:00 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:48:02.822376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014212b-6b79-4c2b-9a2e-41ddcdcdb738 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:02.822949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014212b-6b79-4c2b-9a2e-41ddcdcdb738 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:02.822949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014212b-6b79-4c2b-9a2e-41ddcdcdb738 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:02.824237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 583/1174] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:48:02.830144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace68ff-1231-4f2f-90e5-e9232997f229 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:02.830576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace68ff-1231-4f2f-90e5-e9232997f229 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:02.830680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace68ff-1231-4f2f-90e5-e9232997f229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:02.849246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-bace68ff-1231-4f2f-90e5-e9232997f229 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:48:02.850414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bace68ff-1231-4f2f-90e5-e9232997f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:48:02.892248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:02.916255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 592/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:48:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:48:03.181052 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014212b-6b79-4c2b-9a2e-41ddcdcdb738 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:03.278677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014212b-6b79-4c2b-9a2e-41ddcdcdb738 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:03.303805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:03.314849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc924556-138b-40ed-bedf-8df4460d9178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:03.315479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc924556-138b-40ed-bedf-8df4460d9178 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:03.315754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc924556-138b-40ed-bedf-8df4460d9178 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:03.316025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc924556-138b-40ed-bedf-8df4460d9178 None admin] PATH_INFO: `/v3/domains/baf5ee03ad0f469db6e177d5480d5950` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:03.317171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc924556-138b-40ed-bedf-8df4460d9178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=baf5ee03ad0f469db6e177d5480d5950)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:03.318576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc924556-138b-40ed-bedf-8df4460d9178 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:03.320136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 584/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Oct 3 15:48:03 2022] GET /identity/v3/domains/baf5ee03ad0f469db6e177d5480d5950 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:03.325810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace68ff-1231-4f2f-90e5-e9232997f229 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:03.422303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace68ff-1231-4f2f-90e5-e9232997f229 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:03.454667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ef9e7-ec08-40c4-a353-b16741a57b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:03.455197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ef9e7-ec08-40c4-a353-b16741a57b07 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:03.455474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ef9e7-ec08-40c4-a353-b16741a57b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:03.455744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ef9e7-ec08-40c4-a353-b16741a57b07 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:03.456395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ef9e7-ec08-40c4-a353-b16741a57b07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:03.457870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ef9e7-ec08-40c4-a353-b16741a57b07 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:03.479422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 593/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:48:03 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:48:05.652377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ff1c-8df7-4f3a-bf07-5322c316388b None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:05.652908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ff1c-8df7-4f3a-bf07-5322c316388b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:05.652999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ff1c-8df7-4f3a-bf07-5322c316388b None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:05.654805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 585/1178] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:48:05.660058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e07de-def4-4aa1-9bec-14678ec3a38a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:05.660428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e07de-def4-4aa1-9bec-14678ec3a38a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:05.660610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e07de-def4-4aa1-9bec-14678ec3a38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:05.678650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-494e07de-def4-4aa1-9bec-14678ec3a38a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:48:05.679825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-494e07de-def4-4aa1-9bec-14678ec3a38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:48:05.707068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:05.747136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 594/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:48:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:48:06.017700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ff1c-8df7-4f3a-bf07-5322c316388b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:06.116365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ff1c-8df7-4f3a-bf07-5322c316388b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:06.148861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:06.149473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:06.149753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:06.150025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:06.160564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:48:06.165326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:48:06.166406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:48:06.167248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 586/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:48:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:48:06.173367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e07de-def4-4aa1-9bec-14678ec3a38a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:06.271317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e07de-def4-4aa1-9bec-14678ec3a38a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:06.305927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:06.306448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:06.306724 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:06.306996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:06.307661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:48:06.307963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:06.309390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:06.317788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 595/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:06.323632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:06.375509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:06.419410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17c736-4460-427a-adaa-4ae37b7c9eaf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:06.452247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:06.452882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:06.453156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:06.453424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] PATH_INFO: `/v3/users/1ee416550bff4531913a06e3968aef4b` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:06.465775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee416550bff4531913a06e3968aef4b)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:06.467260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:06.468826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 587/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:48:06 2022] GET /identity/v3/users/1ee416550bff4531913a06e3968aef4b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:06.474797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:06.502715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:06.571688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d916d2-4528-4ab5-97e8-ab8ae8463808 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:06.603778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:06.604363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:06.604635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:06.604941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:06.670655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:06.672112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:06.697245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:06.768394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 596/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:48:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:48:06.774459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:06.870960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf68b0-aee1-4c81-acfa-c82d3f787fc6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:06.882363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:06.905281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7b664-1e7b-4ae4-882b-30084cdbbc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:06.905901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7b664-1e7b-4ae4-882b-30084cdbbc73 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:06.906182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7b664-1e7b-4ae4-882b-30084cdbbc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:06.906456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7b664-1e7b-4ae4-882b-30084cdbbc73 None admin] PATH_INFO: `/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:06.907570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7b664-1e7b-4ae4-882b-30084cdbbc73 None admin] RBAC: Authorizing `identity:get_project(project_id=67b0b7fbaffc429e9c00b630fa09fa55)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:06.909036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7b664-1e7b-4ae4-882b-30084cdbbc73 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:06.910635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 588/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:48:06 2022] GET /identity/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:06.916616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:07.012358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59d8b6-bbed-4d17-a1ee-6d59c94bc770 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:07.036908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:07.046430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17b3cb-d6ef-4150-84d7-ad0181bf1c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:07.047021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17b3cb-d6ef-4150-84d7-ad0181bf1c8f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:07.047293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17b3cb-d6ef-4150-84d7-ad0181bf1c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:07.047558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17b3cb-d6ef-4150-84d7-ad0181bf1c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:07.048279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17b3cb-d6ef-4150-84d7-ad0181bf1c8f None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=67b0b7fbaffc429e9c00b630fa09fa55, user.id=1ee416550bff4531913a06e3968aef4b) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:48:07.048582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17b3cb-d6ef-4150-84d7-ad0181bf1c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:07.050073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17b3cb-d6ef-4150-84d7-ad0181bf1c8f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:07.055326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 597/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:48:06 2022] GET /identity/v3/role_assignments?user.id=1ee416550bff4531913a06e3968aef4b&scope.project.id=67b0b7fbaffc429e9c00b630fa09fa55&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:09.209929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774857-2b3c-428c-8e68-70766cde94c5 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:09.210485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774857-2b3c-428c-8e68-70766cde94c5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:09.210485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef774857-2b3c-428c-8e68-70766cde94c5 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:09.211774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 589/1186] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:48:09.217611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424e21-406a-483f-8347-670365cd47c2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:09.218056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424e21-406a-483f-8347-670365cd47c2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:09.218163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424e21-406a-483f-8347-670365cd47c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:09.236447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e8424e21-406a-483f-8347-670365cd47c2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:48:09.237655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e8424e21-406a-483f-8347-670365cd47c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:48:09.301522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 598/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:48:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:48:09.568451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef774857-2b3c-428c-8e68-70766cde94c5 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:09.666194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef774857-2b3c-428c-8e68-70766cde94c5 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:09.698070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:09.698677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:09.698951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:09.699217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:09.709555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:48:09.714209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:48:09.715367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:48:09.717061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 590/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:48:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:48:09.721224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8424e21-406a-483f-8347-670365cd47c2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:09.818800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8424e21-406a-483f-8347-670365cd47c2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:09.837721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:09.854001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:09.854982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:09.855261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:09.855527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:09.856198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:48:09.856488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:09.858404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:09.866856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 599/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:09.873006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:09.912239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:09.971146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abc7f2-bdba-40b0-9a4b-573d9436b278 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:10.004505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:10.005155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:10.005436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:10.005706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] PATH_INFO: `/v3/users/1ee416550bff4531913a06e3968aef4b` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:10.006851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee416550bff4531913a06e3968aef4b)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:10.008320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:10.009917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 591/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:48:09 2022] GET /identity/v3/users/1ee416550bff4531913a06e3968aef4b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:10.015729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:10.112441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac9a72-640c-4cfb-a23f-51a757591667 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:10.144348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a14d90-4bfe-47c0-b0a2-961b9e46ab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:10.144987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14d90-4bfe-47c0-b0a2-961b9e46ab3f None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:10.145261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14d90-4bfe-47c0-b0a2-961b9e46ab3f None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:10.145525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a14d90-4bfe-47c0-b0a2-961b9e46ab3f None admin] PATH_INFO: `/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:10.146623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a14d90-4bfe-47c0-b0a2-961b9e46ab3f None admin] RBAC: Authorizing `identity:get_project(project_id=67b0b7fbaffc429e9c00b630fa09fa55)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:10.148051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a14d90-4bfe-47c0-b0a2-961b9e46ab3f None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:10.149671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 600/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:48:10 2022] GET /identity/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:10.156475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:10.257141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e407e2-9977-47ed-9d38-dc82b780f423 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:10.263935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:10.290605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab210c1-51f3-452a-99cd-2225c130d706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:10.291254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab210c1-51f3-452a-99cd-2225c130d706 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:10.291530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab210c1-51f3-452a-99cd-2225c130d706 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:10.291800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab210c1-51f3-452a-99cd-2225c130d706 None admin] PATH_INFO: `/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55/users/1ee416550bff4531913a06e3968aef4b/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:10.293720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab210c1-51f3-452a-99cd-2225c130d706 None admin] RBAC: Authorizing `identity:create_grant(project_id=67b0b7fbaffc429e9c00b630fa09fa55, user_id=1ee416550bff4531913a06e3968aef4b, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:10.296366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab210c1-51f3-452a-99cd-2225c130d706 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:10.310790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 592/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:48:10 2022] PUT /identity/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55/users/1ee416550bff4531913a06e3968aef4b/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:48:12.409485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011a909-0341-4f6d-bed8-3ad65d118d0d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:12.409990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011a909-0341-4f6d-bed8-3ad65d118d0d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:12.409990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011a909-0341-4f6d-bed8-3ad65d118d0d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:12.411346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 601/1193] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:48:12.416542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c37954-2d96-4f94-979a-942244f56807 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:12.416981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c37954-2d96-4f94-979a-942244f56807 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:12.417125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c37954-2d96-4f94-979a-942244f56807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:12.436101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d3c37954-2d96-4f94-979a-942244f56807 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:48:12.437324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d3c37954-2d96-4f94-979a-942244f56807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:48:12.502239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 593/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:48:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:48:12.771192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011a909-0341-4f6d-bed8-3ad65d118d0d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:12.821230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:12.872168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011a909-0341-4f6d-bed8-3ad65d118d0d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:12.908643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:12.909298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:12.909577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:12.909850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:12.919971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:48:12.924741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:48:12.925789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:48:12.926995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 602/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:48:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:48:12.932015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c37954-2d96-4f94-979a-942244f56807 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:13.022790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:13.029610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c37954-2d96-4f94-979a-942244f56807 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:13.062429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:13.063010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:13.063282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:13.063546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:13.064190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:48:13.064485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:13.065863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:13.073448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 594/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:48:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:13.080149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:13.177458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f6df3-7104-4b61-8856-fc46480c1bd6 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:13.188130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:13.211254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:13.211868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:13.212139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:13.212403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] PATH_INFO: `/v3/users/1ee416550bff4531913a06e3968aef4b` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:13.213533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] RBAC: Authorizing `identity:get_user(user_id=1ee416550bff4531913a06e3968aef4b)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:13.214971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:13.216555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 603/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Oct 3 15:48:13 2022] GET /identity/v3/users/1ee416550bff4531913a06e3968aef4b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:13.222555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:13.319471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978e57b-727c-4a97-8fcb-e90e9ddff366 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:13.351930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:13.352525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:13.352829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:13.353099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:13.378836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:13.417877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:13.419349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:13.513400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 595/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:48:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:48:13.519725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:13.616361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57884388-88a2-4638-a30f-1bb70710825a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:13.648599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea436d4f-b16b-4306-9776-e4890647dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:13.649236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea436d4f-b16b-4306-9776-e4890647dceb None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:13.649508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea436d4f-b16b-4306-9776-e4890647dceb None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:13.649771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea436d4f-b16b-4306-9776-e4890647dceb None admin] PATH_INFO: `/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:13.650864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea436d4f-b16b-4306-9776-e4890647dceb None admin] RBAC: Authorizing `identity:get_project(project_id=67b0b7fbaffc429e9c00b630fa09fa55)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:13.652300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea436d4f-b16b-4306-9776-e4890647dceb None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:13.654621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 604/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Oct 3 15:48:13 2022] GET /identity/v3/projects/67b0b7fbaffc429e9c00b630fa09fa55 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:13.660684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:48:13.715030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:13.755607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11adca9-854b-4e6f-966e-b899f9c550a5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:48:13.789085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291934bc-75b8-41a4-b4ec-9ee3c174f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:48:13.789673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291934bc-75b8-41a4-b4ec-9ee3c174f9be None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:48:13.789949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291934bc-75b8-41a4-b4ec-9ee3c174f9be None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:48:13.790218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291934bc-75b8-41a4-b4ec-9ee3c174f9be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:48:13.790936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291934bc-75b8-41a4-b4ec-9ee3c174f9be None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=67b0b7fbaffc429e9c00b630fa09fa55, user.id=1ee416550bff4531913a06e3968aef4b) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:48:13.791240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291934bc-75b8-41a4-b4ec-9ee3c174f9be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:48:13.792667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291934bc-75b8-41a4-b4ec-9ee3c174f9be None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:48:13.798290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 596/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:48:13 2022] GET /identity/v3/role_assignments?user.id=1ee416550bff4531913a06e3968aef4b&scope.project.id=67b0b7fbaffc429e9c00b630fa09fa55&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:48:23.644683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=77 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:48:23.795760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=39 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:09.589065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd4fe7-a5e7-4114-84fa-5cb7d6fa454e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:09.589616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd4fe7-a5e7-4114-84fa-5cb7d6fa454e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:09.589616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd4fe7-a5e7-4114-84fa-5cb7d6fa454e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:09.591143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 605/1201] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:09.597483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad27b7f-bdb2-4125-8314-5b842fdc1812 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:09.597939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad27b7f-bdb2-4125-8314-5b842fdc1812 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:09.598033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad27b7f-bdb2-4125-8314-5b842fdc1812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:09.622530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-9ad27b7f-bdb2-4125-8314-5b842fdc1812 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:09.623837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9ad27b7f-bdb2-4125-8314-5b842fdc1812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:09.721763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 597/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:10.030605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd4fe7-a5e7-4114-84fa-5cb7d6fa454e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:10.161671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd4fe7-a5e7-4114-84fa-5cb7d6fa454e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:10.201147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:10.201882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:10.202193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:10.202480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:10.215358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:10.216954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:10.236130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:10.237516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:10.238491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:10.238491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:10.238491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:10.238491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 606/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:10.243847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad27b7f-bdb2-4125-8314-5b842fdc1812 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:10.365435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad27b7f-bdb2-4125-8314-5b842fdc1812 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:10.413876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:10.421782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124a844-badc-4bf9-9ce5-a1a0bc679e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:10.422661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a844-badc-4bf9-9ce5-a1a0bc679e9f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:10.423038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a844-badc-4bf9-9ce5-a1a0bc679e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:10.423364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a844-badc-4bf9-9ce5-a1a0bc679e9f None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:10.424475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124a844-badc-4bf9-9ce5-a1a0bc679e9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:10.424834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124a844-badc-4bf9-9ce5-a1a0bc679e9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:10.426661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124a844-badc-4bf9-9ce5-a1a0bc679e9f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:10.442309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 598/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:10.448085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:10.517581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:10.592345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4d17a-3761-44ff-ac4b-4ae4bf02cdaf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:10.636369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c6084-1939-4189-a673-afc5ebd302ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:10.637138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c6084-1939-4189-a673-afc5ebd302ec None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:10.637502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c6084-1939-4189-a673-afc5ebd302ec None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:10.637772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c6084-1939-4189-a673-afc5ebd302ec None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:10.638455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c6084-1939-4189-a673-afc5ebd302ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:10.640506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c6084-1939-4189-a673-afc5ebd302ec None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:10.669755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 607/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Oct 3 15:49:10 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:49:13.028562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570e3bd-ac3c-4f14-9872-2ed59730f391 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:13.029188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570e3bd-ac3c-4f14-9872-2ed59730f391 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:13.029188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570e3bd-ac3c-4f14-9872-2ed59730f391 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:13.030483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 599/1206] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:13.036141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a59af9-802c-4744-b71d-1a2fe7c5e725 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:13.036573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a59af9-802c-4744-b71d-1a2fe7c5e725 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:13.036875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a59af9-802c-4744-b71d-1a2fe7c5e725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:13.058573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-51a59af9-802c-4744-b71d-1a2fe7c5e725 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:13.060007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-51a59af9-802c-4744-b71d-1a2fe7c5e725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:13.140758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 608/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:13.415568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570e3bd-ac3c-4f14-9872-2ed59730f391 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:13.540416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570e3bd-ac3c-4f14-9872-2ed59730f391 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:13.580237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:13.580812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:13.581091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:13.583385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:13.596010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:49:13.604000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:13.605003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:13.606280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 600/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:49:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:49:13.611868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a59af9-802c-4744-b71d-1a2fe7c5e725 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:13.657552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:13.740787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a59af9-802c-4744-b71d-1a2fe7c5e725 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:13.784774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:13.784774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:13.784774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:13.785192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:13.785686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:13.786035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:13.787446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:13.798006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 609/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:49:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:13.803581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:13.874606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:13.932353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd80fc-653e-40d1-bb10-d84627eb6374 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:13.977442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:13.978072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:13.978349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:13.978799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:13.989828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:13.991337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:14.007385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:14.008370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:14.009386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:14.009386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:14.009386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:14.011883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 601/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:14.017716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:14.132444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:14.136904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a056b27-5d39-40fd-9cc8-d53676cd086a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:14.173867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:14.174527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:14.174884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:14.175200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:14.175937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:14.176285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:14.177785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:14.193044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 610/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:14.200282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:14.309985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6865ede-7030-47e9-85e1-56c8b3f1343f None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:14.311696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:14.348296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:14.349004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:14.349286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:14.349569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] PATH_INFO: `/v3/users/placement` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:14.358062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:14.359652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:49:14.372644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:14.373894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:14.375099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:14.379515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:14.379915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 602/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:14.495549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60806999-0820-4571-97a8-67505ad964c2 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:14.537039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:14.537767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:14.538099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:14.538429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:14.539227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:14.539726 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:14.541779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:14.560682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 611/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 15:49:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:49:14.566382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:14.680074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7eb29-909c-4864-9359-2cb48c24ffde None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:14.725249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:14.726048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:14.726329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:14.726735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:14.762455 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:14.810834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:14.812474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:14.944758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 603/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:49:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:49:14.949492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:14.975772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:15.074395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f17a6d-184a-43d0-98b4-5fa9255358bf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:15.116377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:15.117133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:15.117440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:15.117798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:15.129732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:15.131377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:15.148574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:15.155814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:15.156916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:15.156916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:15.156916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:15.156916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 612/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:15.157197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:15.224995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:15.273475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646741e-f7c8-4d2e-8850-dbe61b76756d None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:15.310466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:15.311072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:15.311401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:15.311611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:15.312257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:15.312554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:15.314076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:15.331316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 604/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:15.332322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:15.427381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:15.464244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8547aa-f2b3-47d6-be32-83eff719299c None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:15.509038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:15.509711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:15.510685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:15.511029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:15.518592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:15.520065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:49:15.532833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:15.534041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:15.534954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 613/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:49:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:15.540202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:15.653458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263d512-71fb-4e91-a29c-9c99a0ffb735 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:15.677907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:15.693872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac1009-7ab8-4774-beeb-f1741fc347ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:15.694434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac1009-7ab8-4774-beeb-f1741fc347ab None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:15.694710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac1009-7ab8-4774-beeb-f1741fc347ab None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:15.695029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac1009-7ab8-4774-beeb-f1741fc347ab None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:15.695724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac1009-7ab8-4774-beeb-f1741fc347ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:15.696102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac1009-7ab8-4774-beeb-f1741fc347ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:15.698423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac1009-7ab8-4774-beeb-f1741fc347ab None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:15.712789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 605/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:49:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:15.718627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:15.842502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca77297-464d-4125-811d-efdb56ccd826 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:15.882175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:15.892106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d7f6c-57b4-408c-9bed-999f6a2d51b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:15.892960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d7f6c-57b4-408c-9bed-999f6a2d51b7 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:15.893342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d7f6c-57b4-408c-9bed-999f6a2d51b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:15.893776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d7f6c-57b4-408c-9bed-999f6a2d51b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:15.894519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d7f6c-57b4-408c-9bed-999f6a2d51b7 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=92a197dc8b2749e08aa0d0612996cb1a) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:15.894867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d7f6c-57b4-408c-9bed-999f6a2d51b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:15.896282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d7f6c-57b4-408c-9bed-999f6a2d51b7 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:15.902703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 614/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:49:15 2022] GET /identity/v3/role_assignments?user.id=92a197dc8b2749e08aa0d0612996cb1a&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:18.092442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dab409-cad7-4852-aae3-7cc908d4f6cf None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:18.093053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dab409-cad7-4852-aae3-7cc908d4f6cf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:18.093053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dab409-cad7-4852-aae3-7cc908d4f6cf None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:18.094538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 606/1220] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:49:18.100464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27e558-bc97-45fc-acc1-b30e32b37273 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:18.101122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27e558-bc97-45fc-acc1-b30e32b37273 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:18.101122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27e558-bc97-45fc-acc1-b30e32b37273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:18.120751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2c27e558-bc97-45fc-acc1-b30e32b37273 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:18.121943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2c27e558-bc97-45fc-acc1-b30e32b37273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:18.200220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 615/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 15:49:18.533153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dab409-cad7-4852-aae3-7cc908d4f6cf None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:18.658720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dab409-cad7-4852-aae3-7cc908d4f6cf None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:18.697998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:18.698642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:18.699116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:18.699484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:18.719025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:49:18.724067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:18.725331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:18.726242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 607/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:18.733240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27e558-bc97-45fc-acc1-b30e32b37273 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:18.843526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27e558-bc97-45fc-acc1-b30e32b37273 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:18.871092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:18.881860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:18.882475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:18.882989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:18.883534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:18.884239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:18.885071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:18.886845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:18.895449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 616/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:49:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:18.902551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:18.949656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:19.012208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35983e36-01ca-49e2-a3b7-3eda7290b26b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:19.048697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:19.049360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:19.049638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:19.049903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:19.061905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:19.063636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:19.080486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:19.081608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:19.082537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:19.082537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:19.082537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:19.082537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 608/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:49:19.088527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:19.197053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0639a0b-eb7e-47fa-a057-c09de6a77b53 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:19.234951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:19.235561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:19.235844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:19.236111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:19.236779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:19.237083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:19.238465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:19.252553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 617/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:19.259200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:19.368245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:19.377249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8dada-ca30-4e26-9baf-be7124587fff None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:19.414811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:19.415501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:19.416036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:19.416327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:19.424493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:19.425972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:49:19.439419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:19.445386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:19.446648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 609/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:19.448149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:19.498841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:19.553022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b51e0b-26ea-43a5-846d-0a77d571d38e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:19.595872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:19.596479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:19.596805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:19.597072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:19.597810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:19.598266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:19.599910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:19.623365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 618/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 15:49:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:19.630716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:19.746698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f272f-291c-4e2c-bded-0640c6c72adc None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:19.781652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:19.782444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:19.782721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:19.783015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:19.794872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:19.796319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:19.813879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:19.815167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:19.816193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:19.816193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:19.816193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:19.816193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 610/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:19.820931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:19.936034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a6ae0-1ed2-4b08-b4ae-663a7db2ec8a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:19.938368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:19.978884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:19.979837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:19.980224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:19.980609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:19.981244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:19.981552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:19.983009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:20.000979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 619/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:20.002204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:20.013334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:20.110305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2478160-9d99-4fc0-9d60-ab907c820a51 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:20.145550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:20.146247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:20.146528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:20.146800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:20.155488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:20.157027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:49:20.170129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:20.171109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:20.172029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 611/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:49:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:49:20.177401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:20.286574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d93ac-e63f-412d-834b-551baffbfc12 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:20.332211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6da76-5c56-4541-bf96-522f2e42b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:20.332883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6da76-5c56-4541-bf96-522f2e42b8ee None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:20.333156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6da76-5c56-4541-bf96-522f2e42b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:20.333426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6da76-5c56-4541-bf96-522f2e42b8ee None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:20.334131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6da76-5c56-4541-bf96-522f2e42b8ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:20.334438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6da76-5c56-4541-bf96-522f2e42b8ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:20.336138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6da76-5c56-4541-bf96-522f2e42b8ee None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:20.351375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 620/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:49:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:20.357898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:20.428101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:20.464568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14020660-f429-4a96-bb98-6d538f54f5b3 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:20.500018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5b2a0-ba1f-4223-929f-7543bb4d6462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:20.500674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5b2a0-ba1f-4223-929f-7543bb4d6462 None admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:20.501036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5b2a0-ba1f-4223-929f-7543bb4d6462 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:20.501319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5b2a0-ba1f-4223-929f-7543bb4d6462 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/92a197dc8b2749e08aa0d0612996cb1a/roles/e3ae1cd82e064f38a35cc76b525ffbbf` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:20.516357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5b2a0-ba1f-4223-929f-7543bb4d6462 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=92a197dc8b2749e08aa0d0612996cb1a, role_id=e3ae1cd82e064f38a35cc76b525ffbbf)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:20.517845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5b2a0-ba1f-4223-929f-7543bb4d6462 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:20.531380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 612/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:49:20 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/92a197dc8b2749e08aa0d0612996cb1a/roles/e3ae1cd82e064f38a35cc76b525ffbbf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:49:22.847279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400694c-a93e-4ed7-9957-e984485cd3bf None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:22.847853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400694c-a93e-4ed7-9957-e984485cd3bf None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:22.847853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400694c-a93e-4ed7-9957-e984485cd3bf None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:22.849398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 621/1233] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:22.855756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605c72-0992-4513-bf23-975993cec0d7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:22.856210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605c72-0992-4513-bf23-975993cec0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:22.856481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605c72-0992-4513-bf23-975993cec0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:22.875799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-01605c72-0992-4513-bf23-975993cec0d7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:22.877002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-01605c72-0992-4513-bf23-975993cec0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:22.967917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 613/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:23.235611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400694c-a93e-4ed7-9957-e984485cd3bf None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:23.264149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:23.358219 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400694c-a93e-4ed7-9957-e984485cd3bf None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:23.397173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:23.398009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:23.398310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:23.398629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:23.409797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:49:23.414749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:23.416621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:23.417660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 622/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Oct 3 15:49:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 15:49:23.422566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605c72-0992-4513-bf23-975993cec0d7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:23.474667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:23.537652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605c72-0992-4513-bf23-975993cec0d7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:23.571944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:23.572548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:23.572853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:23.573122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:23.573772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] RBAC: Adding query filter params (name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:23.574066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:23.575420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:23.583141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 614/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Oct 3 15:49:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:23.589563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:23.682988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:23.701014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56707e79-2070-4e16-aeaf-e204973db1b1 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:23.742623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:23.743385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:23.743773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:23.744048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:23.754493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:23.755970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:23.773175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:23.774301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:23.776212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:23.776212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:23.776212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:23.776212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 623/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:23.781403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:23.893210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0443a-8bf8-4867-9569-677ac170586e None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:23.904508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:23.934836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:23.935518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:23.935786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:23.936141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:23.936973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:23.937262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:23.938933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:23.954599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 615/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:23.960948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:24.081551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672d85d-1e17-44b1-a966-f61dd2a50be0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:24.119608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:24.120328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:24.120644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:24.120992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:24.124073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:24.129702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:24.131262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:49:24.148226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:24.149341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:24.151273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 624/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:24.157711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:24.262947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba54f13-09ec-4ddd-97f3-8ea9ee42bfaf None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:24.300985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:24.301595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:24.301865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:24.302133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:24.302856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:24.303157 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:24.304581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:24.326282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 616/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 15:49:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:24.333058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:24.446142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9a68e-c3fc-45c4-ab04-f70806478911 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:24.486174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:24.486895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:24.487286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:24.487581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:24.565053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:24.572935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:24.574711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:24.690459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 625/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:49:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:49:24.696956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:24.728797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:24.812417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf965-3bf1-4175-ac40-921b129cc18b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:24.848678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:24.849324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:24.849599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:24.849875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:24.861123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:24.862589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:24.878712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:24.879855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:24.880998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:24.880998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:24.880998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:24.880998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 617/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:24.887779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:25.011034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:25.018651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756c99c-6e85-45d6-a26c-75663c233e91 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:25.053800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:25.054433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:25.054766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:25.055090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:25.055766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:25.056097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:25.057655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:25.070511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 626/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:25.076357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:25.164632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:25.188106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45f27b-a3ac-497a-adf0-c1c4685ce985 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:25.225436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:25.226197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:25.226469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:25.226708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:25.234708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:25.236148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:49:25.250486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:25.251790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:25.252989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 618/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:49:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:25.257747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:25.382943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3bd4dc-abf7-4333-825a-39cdae21b31d None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:25.422066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e9f8d-ae5a-4a68-81bf-50fc104cface None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:25.422769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e9f8d-ae5a-4a68-81bf-50fc104cface None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:25.423044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e9f8d-ae5a-4a68-81bf-50fc104cface None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:25.423355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e9f8d-ae5a-4a68-81bf-50fc104cface None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:25.424264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e9f8d-ae5a-4a68-81bf-50fc104cface None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:25.424569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e9f8d-ae5a-4a68-81bf-50fc104cface None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:25.426364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e9f8d-ae5a-4a68-81bf-50fc104cface None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:25.440529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 627/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:49:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:25.447776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:25.566720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df706a-c63b-4f7e-9298-1ae9b11f27d2 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:25.606095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a6943-8e63-45f9-a525-21babc3f02c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:25.606674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6943-8e63-45f9-a525-21babc3f02c3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:25.606957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6943-8e63-45f9-a525-21babc3f02c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:25.607328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a6943-8e63-45f9-a525-21babc3f02c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:25.608182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a6943-8e63-45f9-a525-21babc3f02c3 None admin] RBAC: Adding query filter params (role.id=e3ae1cd82e064f38a35cc76b525ffbbf, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=92a197dc8b2749e08aa0d0612996cb1a) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:25.608489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a6943-8e63-45f9-a525-21babc3f02c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:25.610142 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a6943-8e63-45f9-a525-21babc3f02c3 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:25.617271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 619/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:49:25 2022] GET /identity/v3/role_assignments?user.id=92a197dc8b2749e08aa0d0612996cb1a&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=e3ae1cd82e064f38a35cc76b525ffbbf => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:28.013764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde392b-0566-44b6-83ae-288e928a4ce9 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:28.014420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde392b-0566-44b6-83ae-288e928a4ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:28.014420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde392b-0566-44b6-83ae-288e928a4ce9 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:28.015640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 628/1247] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:28.021246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a10df-214e-4f71-9d1b-14b35fd6302e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:28.021774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a10df-214e-4f71-9d1b-14b35fd6302e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:28.021774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a10df-214e-4f71-9d1b-14b35fd6302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:28.028587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:28.042135 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-844a10df-214e-4f71-9d1b-14b35fd6302e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:28.043314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-844a10df-214e-4f71-9d1b-14b35fd6302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:28.116857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 620/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:28.408811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde392b-0566-44b6-83ae-288e928a4ce9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:28.428784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:28.519987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde392b-0566-44b6-83ae-288e928a4ce9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:28.558803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:28.559439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:28.559744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:28.560055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:28.570795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:49:28.577800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:28.579698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:28.580573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 629/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:28.585474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a10df-214e-4f71-9d1b-14b35fd6302e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:28.696937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a10df-214e-4f71-9d1b-14b35fd6302e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:28.731364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:28.737160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:28.737867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:28.738186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:28.738462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:28.739130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:28.739484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:28.741144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:28.750087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 621/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:49:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:28.756172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:28.849643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:28.878183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb749c5-cc67-4204-970d-d29aa9e0bac3 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:28.918599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:28.919252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:28.919529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:28.919794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:28.931338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:28.932926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:28.949554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:28.950885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:28.952026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:28.952026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:28.952026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:28.952026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 630/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:28.957036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:29.067013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c474c-3e48-4937-97f9-3202f06f7312 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:29.102285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:29.102948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:29.103270 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:29.103542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:29.104197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:29.104498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:29.106432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:29.119160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 622/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:29.125015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:29.232026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753f531-e3e9-4516-a33a-547293cf4194 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:29.260697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:29.271487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:29.271487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:29.271487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:29.271487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:29.279220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:29.280704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:49:29.292182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:29.293355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:29.294549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 631/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:29.301347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:29.360671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:29.405115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96920dd5-2278-4126-a81a-a236ceb518cd None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:29.439672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:29.440289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:29.440568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:29.440923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:29.441667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:29.441976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:29.443400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:29.461137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 623/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 15:49:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:29.467068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:29.570382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661189b8-da9e-4df9-8741-cf64c8c9500a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:29.603937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:29.604528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:29.604833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:29.605105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:29.657768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:29.670664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:29.672124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:29.768590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 632/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:49:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:49:29.774390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:29.871619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203c4bb-02c5-4aad-89d2-6f56e5066a59 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:29.884245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:29.905388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:29.906005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:29.906281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:29.906549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:29.916957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:29.918342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:29.932482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:29.933578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:29.934502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:29.934502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:29.934502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:29.934502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 624/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:29.942169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:30.037517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:30.049458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489117df-f69d-4e18-b658-176a3f050bb8 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:30.082112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:30.082700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:30.082972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:30.083235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:30.083866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:30.084160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:30.085546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:30.097500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 633/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:30.103222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:30.203671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7d0cc-fb06-473b-bd4d-e0a152829ca3 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:30.238425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:30.239037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:30.239316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:30.239585 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:30.246074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:30.247520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:49:30.258426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:30.259892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:30.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 625/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:30.266817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:30.366861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e7ff4-71d1-4d13-a1fb-28a3040e11b9 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:30.400577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2beaad-e0d1-44d8-8866-021ef3dc2311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:30.401211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2beaad-e0d1-44d8-8866-021ef3dc2311 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:30.401489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2beaad-e0d1-44d8-8866-021ef3dc2311 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:30.401758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2beaad-e0d1-44d8-8866-021ef3dc2311 None admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:30.402444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2beaad-e0d1-44d8-8866-021ef3dc2311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:30.402746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2beaad-e0d1-44d8-8866-021ef3dc2311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:30.404181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2beaad-e0d1-44d8-8866-021ef3dc2311 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:30.415823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 634/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:30.422046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:30.448290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:30.522061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f8fca-7798-4b00-aa93-7517574c84f6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:30.553515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd8ad2-d02b-4534-9c77-40250fa3b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:30.554094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd8ad2-d02b-4534-9c77-40250fa3b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:30.554363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd8ad2-d02b-4534-9c77-40250fa3b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:30.554624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd8ad2-d02b-4534-9c77-40250fa3b6c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:30.555335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd8ad2-d02b-4534-9c77-40250fa3b6c9 None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=92a197dc8b2749e08aa0d0612996cb1a) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:30.555636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd8ad2-d02b-4534-9c77-40250fa3b6c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:30.557597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd8ad2-d02b-4534-9c77-40250fa3b6c9 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:30.562744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 626/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:49:30 2022] GET /identity/v3/role_assignments?user.id=92a197dc8b2749e08aa0d0612996cb1a&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=0cb19d985d3d456983e0b1624610b058 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:32.704298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197103-29fd-4eda-aecc-eb87a378ad45 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:32.704840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197103-29fd-4eda-aecc-eb87a378ad45 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:32.704840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197103-29fd-4eda-aecc-eb87a378ad45 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:32.706119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 635/1261] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:32.711450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2c35f-a422-4fcb-a0c7-ceff63c0791a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:32.712608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2c35f-a422-4fcb-a0c7-ceff63c0791a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:32.712608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2c35f-a422-4fcb-a0c7-ceff63c0791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:32.736701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0ae2c35f-a422-4fcb-a0c7-ceff63c0791a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:32.737955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0ae2c35f-a422-4fcb-a0c7-ceff63c0791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:32.812999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 627/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:33.149416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197103-29fd-4eda-aecc-eb87a378ad45 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:33.165055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:33.262101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197103-29fd-4eda-aecc-eb87a378ad45 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:33.295182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:33.295796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:33.296075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:33.296345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:33.306375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:49:33.311082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:33.312173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:33.313162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 636/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:49:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:33.318177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2c35f-a422-4fcb-a0c7-ceff63c0791a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:33.336442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:33.417973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2c35f-a422-4fcb-a0c7-ceff63c0791a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:33.451406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:33.452010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:33.452285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:33.452551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:33.453244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:33.453540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:33.454893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:33.462819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 628/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:49:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:33.468605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:33.539035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:33.572834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc3269-004d-4b36-b184-dae4fbc717ba None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:33.605880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:33.606504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:33.606783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:33.607052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:33.616773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:33.618744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:33.632791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:33.634507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:33.635440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:33.635440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:33.635440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:33.635440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 637/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:33.640432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:33.718341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:33.737945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c9922-760c-4f45-adb7-f971be7d69a4 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:33.770075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:33.770661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:33.770939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:33.771207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:33.771838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:33.772137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:33.773524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:33.784705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 629/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:33.790893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:33.889591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b38cc-3802-4114-9b07-b754938e941a None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:33.908850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:33.923645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:33.924297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:33.924570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:33.924868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:33.932893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:33.934585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:49:33.945876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:33.948396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:33.949366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 638/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:33.953423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:34.052779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6ef49-7e16-41db-8cbf-b3b655f9500b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:34.089304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:34.089886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:34.090154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:34.090416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:34.091121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:34.091414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:34.092845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:34.101361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:34.111289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 630/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 15:49:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:34.117693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:34.215086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1e16f-46f6-4e83-ba06-5e743768314e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:34.247681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:34.248285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:34.248560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:34.248853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:34.258893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:34.260325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:34.273803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:34.275401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:34.276558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:34.276558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:34.276558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:34.276558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 639/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:34.281300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:34.377906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96d908-5aad-4878-bc71-d4c09c48bc59 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:34.410235 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:34.410824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:34.411097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:34.411360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:34.411986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:34.412278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:34.413651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:34.424812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 631/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:34.430625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:34.460763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:34.529601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16697bad-d94f-4ce1-9ff0-dc5e1cbcbccd None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:34.561830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:34.562447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:34.562730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:34.562998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:34.569537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:34.570923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:49:34.583049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:34.584081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:34.585165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 640/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:34.590637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:34.651226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:34.691552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e7aae-d4d7-43c5-b1da-b6e3787737ea None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:34.725154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1e4f2-86c7-4f6a-b293-d098a0ad80b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:34.725750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e4f2-86c7-4f6a-b293-d098a0ad80b1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:34.726027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e4f2-86c7-4f6a-b293-d098a0ad80b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:34.726297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1e4f2-86c7-4f6a-b293-d098a0ad80b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:34.726981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1e4f2-86c7-4f6a-b293-d098a0ad80b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:34.727284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1e4f2-86c7-4f6a-b293-d098a0ad80b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:34.728963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1e4f2-86c7-4f6a-b293-d098a0ad80b1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:34.740536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 632/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:34.746674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:34.834314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:34.843952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c17736-4705-43f3-875b-f6fbe8d7b1f0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:34.877071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af0d91-5806-4061-8d43-8b24d98d93b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:34.877755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af0d91-5806-4061-8d43-8b24d98d93b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:34.877983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af0d91-5806-4061-8d43-8b24d98d93b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:34.878258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af0d91-5806-4061-8d43-8b24d98d93b8 None admin] PATH_INFO: `/v3/projects/78c46fd4a4864995b6630028c672c630/users/92a197dc8b2749e08aa0d0612996cb1a/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:34.880150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af0d91-5806-4061-8d43-8b24d98d93b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=78c46fd4a4864995b6630028c672c630, user_id=92a197dc8b2749e08aa0d0612996cb1a, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:34.881623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af0d91-5806-4061-8d43-8b24d98d93b8 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:34.894466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 641/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Oct 3 15:49:34 2022] PUT /identity/v3/projects/78c46fd4a4864995b6630028c672c630/users/92a197dc8b2749e08aa0d0612996cb1a/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:49:37.017307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2f6d1-440a-4e77-a6b6-2255ea305b7f None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:37.017841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2f6d1-440a-4e77-a6b6-2255ea305b7f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:37.017841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2f6d1-440a-4e77-a6b6-2255ea305b7f None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:37.019149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 633/1274] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:37.024259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae7b8-30dc-4215-875d-46349eee1ab3 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:37.024690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae7b8-30dc-4215-875d-46349eee1ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:37.024824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ae7b8-30dc-4215-875d-46349eee1ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:37.043776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6f4ae7b8-30dc-4215-875d-46349eee1ab3 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:37.045662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6f4ae7b8-30dc-4215-875d-46349eee1ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:37.111412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 642/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:37.387723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2f6d1-440a-4e77-a6b6-2255ea305b7f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:37.485928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2f6d1-440a-4e77-a6b6-2255ea305b7f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:37.502140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:37.518677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:37.519296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:37.519572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:37.519839 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:37.530134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 15:49:37.534827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:37.535914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:37.537192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 634/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Oct 3 15:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:37.541881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae7b8-30dc-4215-875d-46349eee1ab3 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:37.620657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:37.639340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ae7b8-30dc-4215-875d-46349eee1ab3 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:37.672530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:37.673146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:37.673424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:37.673694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:37.674350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:37.674650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:37.676025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:37.683918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 643/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Oct 3 15:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:37.689630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:37.787724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27563ee-b66b-4db7-bfdb-1deabe6d37e6 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:37.820326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:37.821537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:37.821787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:37.822216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:37.831745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:37.833530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:37.849393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:37.850554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:37.851653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:37.851653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:37.851653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:37.851653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 635/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:37.856241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:37.960513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127098b-a3fa-4b9d-baa6-4be26c67114e None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:38.000468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:38.001167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:38.001491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:38.001867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:38.002574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:38.002947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:38.004497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:38.018432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 644/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:38.024609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:38.061094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:38.128774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70bc4f-2956-4a8b-b363-4b1aa64b444b None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:38.161507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:38.162123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:38.162400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:38.162671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:38.169992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:38.171442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 15:49:38.182840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:38.183916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:38.185082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 636/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:38.189876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:38.193586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:38.292948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85a391-ee7f-4749-b21a-d38205a5fabf None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:38.328342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:38.328984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:38.329260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:38.329524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:38.330245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:38.330546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:38.331967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:38.348712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 645/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Oct 3 15:49:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:38.355151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:38.442400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:38.452045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d0bc-4f08-4279-9cb6-9e1b1fed6bd0 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:38.484762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:38.485353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:38.485628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:38.485896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:38.551784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:38.553191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:38.640902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:38.653249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 637/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Oct 3 15:49:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 15:49:38.659237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:38.720818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:38.758497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1a1a3-e34e-4744-8c35-43bf70ff5704 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:38.792005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:38.792613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:38.792912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:38.793177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:38.802854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:38.804232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 15:49:38.818221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:38.819298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 15:49:38.820324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 15:49:38.820324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 15:49:38.820324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:38.820324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 646/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:38.825066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:38.923359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bb401-394d-4650-a029-036de402b865 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:38.956640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:38.957263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:38.957538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:38.957807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:38.958448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:38.958749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:38.960114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:38.972000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 638/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Oct 3 15:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:38.978424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:39.074480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afa768-f513-4997-8c76-6dd70ba56371 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:39.088515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:39.108198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:39.108842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:39.109117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:39.109384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:39.115738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:39.117221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 15:49:39.128699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:39.129612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: ERROR keystone.server.flask.application Oct 03 15:49:39.130643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 647/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Oct 3 15:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:39.136258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:39.184429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:39.234385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4aa33-6b53-4d73-bc97-87f1512fefd1 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:39.266664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1893f4-d322-4ff3-aef7-1b22a8c4315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:39.267250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1893f4-d322-4ff3-aef7-1b22a8c4315f None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:39.267526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1893f4-d322-4ff3-aef7-1b22a8c4315f None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:39.267796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1893f4-d322-4ff3-aef7-1b22a8c4315f None admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:39.268477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1893f4-d322-4ff3-aef7-1b22a8c4315f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:39.268808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1893f4-d322-4ff3-aef7-1b22a8c4315f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:39.270233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1893f4-d322-4ff3-aef7-1b22a8c4315f None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:39.282279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 639/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Oct 3 15:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:39.288327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:39.386740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c20c9-d060-4c1a-a276-e785904d70c0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:39.419004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f0c23-faec-48b5-b2c5-bc73c5779a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:39.419596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f0c23-faec-48b5-b2c5-bc73c5779a8c None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:39.419871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f0c23-faec-48b5-b2c5-bc73c5779a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:39.420140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f0c23-faec-48b5-b2c5-bc73c5779a8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:39.420899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f0c23-faec-48b5-b2c5-bc73c5779a8c None admin] RBAC: Adding query filter params (role.id=0cb19d985d3d456983e0b1624610b058, scope.project.id=78c46fd4a4864995b6630028c672c630, user.id=92a197dc8b2749e08aa0d0612996cb1a) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:39.421203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f0c23-faec-48b5-b2c5-bc73c5779a8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:39.422640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f0c23-faec-48b5-b2c5-bc73c5779a8c None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:39.428295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 648/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Oct 3 15:49:39 2022] GET /identity/v3/role_assignments?user.id=92a197dc8b2749e08aa0d0612996cb1a&scope.project.id=78c46fd4a4864995b6630028c672c630&role.id=0cb19d985d3d456983e0b1624610b058 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:41.649817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0984-8ff6-4d48-adf7-721b48e8a88e None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:41.650406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0984-8ff6-4d48-adf7-721b48e8a88e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:41.650406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0984-8ff6-4d48-adf7-721b48e8a88e None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:41.651573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 640/1288] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:41.656678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10508edb-f3ce-44fe-93fc-a605f13e51fa None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:41.657114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10508edb-f3ce-44fe-93fc-a605f13e51fa None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:41.657273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10508edb-f3ce-44fe-93fc-a605f13e51fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:41.675763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-10508edb-f3ce-44fe-93fc-a605f13e51fa None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:41.676991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-10508edb-f3ce-44fe-93fc-a605f13e51fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:41.705671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:41.742641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 649/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:42.017282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b0984-8ff6-4d48-adf7-721b48e8a88e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:42.115088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b0984-8ff6-4d48-adf7-721b48e8a88e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:42.126182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:42.149025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:42.149728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:42.150003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:42.150271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:42.150844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:42.152200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:42.160661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:42.161865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:42.162869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 641/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:49:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:42.168273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10508edb-f3ce-44fe-93fc-a605f13e51fa None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:42.264679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10508edb-f3ce-44fe-93fc-a605f13e51fa None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:42.297267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179547f-1114-4a6d-9206-aa32099837eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:42.297793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179547f-1114-4a6d-9206-aa32099837eb None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:42.298065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179547f-1114-4a6d-9206-aa32099837eb None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:42.298367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179547f-1114-4a6d-9206-aa32099837eb None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:42.298987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179547f-1114-4a6d-9206-aa32099837eb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:42.299282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179547f-1114-4a6d-9206-aa32099837eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:42.300654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179547f-1114-4a6d-9206-aa32099837eb None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:42.306254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 650/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 15:49:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:42.311932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:42.408456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97035df8-c0ae-442c-8202-09d2a990cd57 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:42.441277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116d11b-30db-41b3-a20f-c25b0e6dccc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:42.441865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116d11b-30db-41b3-a20f-c25b0e6dccc5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:42.442138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116d11b-30db-41b3-a20f-c25b0e6dccc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:42.442402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116d11b-30db-41b3-a20f-c25b0e6dccc5 None admin] PATH_INFO: `/v3/services` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:42.443023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116d11b-30db-41b3-a20f-c25b0e6dccc5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:42.443319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116d11b-30db-41b3-a20f-c25b0e6dccc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:42.444668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116d11b-30db-41b3-a20f-c25b0e6dccc5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:42.449011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 642/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 15:49:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:44.604299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bea120-c3c9-4e11-9b97-aa70021e898c None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:44.604833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bea120-c3c9-4e11-9b97-aa70021e898c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:44.604833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bea120-c3c9-4e11-9b97-aa70021e898c None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:44.606152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 651/1293] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:44.611195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a020f-4493-465d-9cb1-348aa9ad7a8a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:44.611584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a020f-4493-465d-9cb1-348aa9ad7a8a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:44.611728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a020f-4493-465d-9cb1-348aa9ad7a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:44.629962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-285a020f-4493-465d-9cb1-348aa9ad7a8a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:44.631119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-285a020f-4493-465d-9cb1-348aa9ad7a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:44.663647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:44.700477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 643/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:44.968510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bea120-c3c9-4e11-9b97-aa70021e898c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:44.999961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:45.071387 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bea120-c3c9-4e11-9b97-aa70021e898c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:45.104093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ff0ec-2a7e-4fa4-b30c-4659073423c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:45.104690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ff0ec-2a7e-4fa4-b30c-4659073423c1 None admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:45.104991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ff0ec-2a7e-4fa4-b30c-4659073423c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:45.105260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ff0ec-2a7e-4fa4-b30c-4659073423c1 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:45.105922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ff0ec-2a7e-4fa4-b30c-4659073423c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:45.107323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ff0ec-2a7e-4fa4-b30c-4659073423c1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:45.122158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 652/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Oct 3 15:49:44 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:49:47.240065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944aee-1796-4d0e-a095-ea8b44441ad7 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:47.240628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944aee-1796-4d0e-a095-ea8b44441ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:47.240628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944aee-1796-4d0e-a095-ea8b44441ad7 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:47.241959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 644/1296] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:47.247382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a793a5b-7b65-49ad-9c49-81658e28ca89 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:47.247761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a793a5b-7b65-49ad-9c49-81658e28ca89 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:47.247932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a793a5b-7b65-49ad-9c49-81658e28ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:47.266673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2a793a5b-7b65-49ad-9c49-81658e28ca89 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:47.267867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2a793a5b-7b65-49ad-9c49-81658e28ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:47.341160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 653/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:47 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:47.604408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944aee-1796-4d0e-a095-ea8b44441ad7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:47.704481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944aee-1796-4d0e-a095-ea8b44441ad7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:47.737646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:47.738272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:47.738550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:47.738817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:47.739390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:47.740776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:47.749912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:47.751150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:47.752036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:47.756065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 645/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:49:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:47.757118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a793a5b-7b65-49ad-9c49-81658e28ca89 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:47.776047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:47.856196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a793a5b-7b65-49ad-9c49-81658e28ca89 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:47.889721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:47.890309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:47.890580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:47.890845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:47.891464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:47.891758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:47.893160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:47.898101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 654/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 15:49:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:49:47.904340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:47.935277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:48.017177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d45a-e949-491d-b8ab-123540954955 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:48.049990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f4bc9-3b97-4b33-a710-0838d097eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:48.050584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f4bc9-3b97-4b33-a710-0838d097eab2 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:48.050860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f4bc9-3b97-4b33-a710-0838d097eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:48.051129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f4bc9-3b97-4b33-a710-0838d097eab2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:48.051822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f4bc9-3b97-4b33-a710-0838d097eab2 None admin] RBAC: Adding query filter params (interface=public, service_id=6b9a9ac964484ee6a15513d2d621a5c4, region_id=RegionOne) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:48.052125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f4bc9-3b97-4b33-a710-0838d097eab2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:48.053515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f4bc9-3b97-4b33-a710-0838d097eab2 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:48.059243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 646/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Oct 3 15:49:47 2022] GET /identity/v3/endpoints?service_id=6b9a9ac964484ee6a15513d2d621a5c4&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:48.064899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:48.144849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:48.162866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1635f3-3d6c-46ff-87bc-2799143a7ce0 None admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:48.196908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ba1d7-5574-414b-b43a-7d08d7017cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:48.197497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ba1d7-5574-414b-b43a-7d08d7017cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:48.197776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ba1d7-5574-414b-b43a-7d08d7017cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:48.198045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ba1d7-5574-414b-b43a-7d08d7017cc1 None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:48.198648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ba1d7-5574-414b-b43a-7d08d7017cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:48.200006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ba1d7-5574-414b-b43a-7d08d7017cc1 None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:48.206579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 655/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Oct 3 15:49:48 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:49:50.379952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21f70-8878-4aeb-8456-8491df8abee4 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:50.380484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21f70-8878-4aeb-8456-8491df8abee4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:50.380484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21f70-8878-4aeb-8456-8491df8abee4 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:50.381868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 647/1302] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:49:50.387268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f4d72-3f77-4ad7-bf71-35e7915077ca None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:50.387674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f4d72-3f77-4ad7-bf71-35e7915077ca None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:50.387812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f4d72-3f77-4ad7-bf71-35e7915077ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:50.405875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-283f4d72-3f77-4ad7-bf71-35e7915077ca None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:49:50.407042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-283f4d72-3f77-4ad7-bf71-35e7915077ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:49:50.470953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 656/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:49:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 15:49:50.745552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21f70-8878-4aeb-8456-8491df8abee4 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:50.839659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:50.846912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21f70-8878-4aeb-8456-8491df8abee4 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:50.879627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:50.880249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:50.880520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:50.880817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:50.881396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:50.882807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 15:49:50.891447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 15:49:50.892819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application with Lock( Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self._enter() Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application return self.creator() Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application created_value = creator( Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: ERROR keystone.server.flask.application Oct 03 15:49:50.894018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 648/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Oct 3 15:49:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 15:49:50.899451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f4d72-3f77-4ad7-bf71-35e7915077ca None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:50.977981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:49:51.001010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f4d72-3f77-4ad7-bf71-35e7915077ca None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:51.036053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7eea00-27e9-4582-8e07-d329f89b3ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:51.036653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eea00-27e9-4582-8e07-d329f89b3ffc None admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:51.036956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eea00-27e9-4582-8e07-d329f89b3ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:51.037222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7eea00-27e9-4582-8e07-d329f89b3ffc None admin] PATH_INFO: `/v3/services` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:51.037845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7eea00-27e9-4582-8e07-d329f89b3ffc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:49:51.038142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7eea00-27e9-4582-8e07-d329f89b3ffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:51.040036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7eea00-27e9-4582-8e07-d329f89b3ffc None admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:51.045005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 657/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Oct 3 15:49:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:49:51.050775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:49:51.150469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e362e-1189-4182-87c5-92c2729de6c5 None admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:49:51.182443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4694ab5-5366-4390-acb8-bda9cd6f2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:49:51.183036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4694ab5-5366-4390-acb8-bda9cd6f2c4c None admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:49:51.183314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4694ab5-5366-4390-acb8-bda9cd6f2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:49:51.183585 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4694ab5-5366-4390-acb8-bda9cd6f2c4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:49:51.184240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4694ab5-5366-4390-acb8-bda9cd6f2c4c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:49:51.185617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4694ab5-5366-4390-acb8-bda9cd6f2c4c None admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:49:51.204870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 649/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:49:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:50:01.042109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=30 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:50:01.200547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:50:55.244328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69bc3-3a70-4c60-a660-9437b9c6f3ee None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:55.245431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69bc3-3a70-4c60-a660-9437b9c6f3ee None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:55.245957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69bc3-3a70-4c60-a660-9437b9c6f3ee None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:55.251510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 658/1307] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:50:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:50:55.252897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc8021-fa2b-4cb3-9aa4-d816167814f7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:55.253322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc8021-fa2b-4cb3-9aa4-d816167814f7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:55.253524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc8021-fa2b-4cb3-9aa4-d816167814f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:55.361620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-89fc8021-fa2b-4cb3-9aa4-d816167814f7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:50:55.362909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-89fc8021-fa2b-4cb3-9aa4-d816167814f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:50:55.512878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 650/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:50:55.805755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e1b79-3f48-496f-b6dd-8697ebb4f26e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:55.806039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e1b79-3f48-496f-b6dd-8697ebb4f26e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:55.806511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e1b79-3f48-496f-b6dd-8697ebb4f26e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:55.807852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 659/1309] 10.222.0.33 () {54 vars in 964 bytes} [Mon Oct 3 15:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:50:55.813898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808ad75-5366-4f51-b2b2-c721bc2ee485 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:55.814182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808ad75-5366-4f51-b2b2-c721bc2ee485 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:55.814448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808ad75-5366-4f51-b2b2-c721bc2ee485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:55.903856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f808ad75-5366-4f51-b2b2-c721bc2ee485 None None] MFA Rules not processed for user `2d007a3d205345aebcc36a36d039da2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:50:55.905373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f808ad75-5366-4f51-b2b2-c721bc2ee485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:50:55.929250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:50:56.003385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 651/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Oct 3 15:50:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:50:56.018014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e1b79-3f48-496f-b6dd-8697ebb4f26e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:50:56.023882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-403e1b79-3f48-496f-b6dd-8697ebb4f26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:50:56.150513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e1b79-3f48-496f-b6dd-8697ebb4f26e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:50:56.232186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f9308-3181-4a5c-8f24-c84bb8e35425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:50:56.232831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f9308-3181-4a5c-8f24-c84bb8e35425 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:56.233105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f9308-3181-4a5c-8f24-c84bb8e35425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:56.233365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f9308-3181-4a5c-8f24-c84bb8e35425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:56.235076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1e8f9308-3181-4a5c-8f24-c84bb8e35425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:50:56.296328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:50:56.361450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f9308-3181-4a5c-8f24-c84bb8e35425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:50:56.362900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f9308-3181-4a5c-8f24-c84bb8e35425 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:50:56.382457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 660/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:50:58.893661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373b19-2e5b-4e57-bf7a-93c6977fba95 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:58.894191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373b19-2e5b-4e57-bf7a-93c6977fba95 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:58.894191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373b19-2e5b-4e57-bf7a-93c6977fba95 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:58.895709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 652/1312] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:50:58.900901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f109d2a-3bc9-4a4a-8f36-b2168d46a993 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:58.901301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f109d2a-3bc9-4a4a-8f36-b2168d46a993 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:58.901447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f109d2a-3bc9-4a4a-8f36-b2168d46a993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:58.971864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-3f109d2a-3bc9-4a4a-8f36-b2168d46a993 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:50:58.973228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3f109d2a-3bc9-4a4a-8f36-b2168d46a993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:50:59.069164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 661/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:50:59.362169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373b19-2e5b-4e57-bf7a-93c6977fba95 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:50:59.364687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cd373b19-2e5b-4e57-bf7a-93c6977fba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:50:59.470281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373b19-2e5b-4e57-bf7a-93c6977fba95 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:50:59.541866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:50:59.544414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893b4b1-ab4b-4aa1-a6dd-3062a4ffcc1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:50:59.545236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b4b1-ab4b-4aa1-a6dd-3062a4ffcc1b service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:50:59.545551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b4b1-ab4b-4aa1-a6dd-3062a4ffcc1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:50:59.546018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893b4b1-ab4b-4aa1-a6dd-3062a4ffcc1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:50:59.547917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c893b4b1-ab4b-4aa1-a6dd-3062a4ffcc1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:50:59.655958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893b4b1-ab4b-4aa1-a6dd-3062a4ffcc1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:50:59.657418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893b4b1-ab4b-4aa1-a6dd-3062a4ffcc1b service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:50:59.677121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 653/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:02.157630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee74668-c262-40bb-b1e0-30c211a1afa0 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:02.158176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee74668-c262-40bb-b1e0-30c211a1afa0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:02.158176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee74668-c262-40bb-b1e0-30c211a1afa0 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:02.159533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 662/1315] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:02.164861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df00c7-c046-473c-955d-e5aa9eeacf35 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:02.165279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df00c7-c046-473c-955d-e5aa9eeacf35 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:02.165402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df00c7-c046-473c-955d-e5aa9eeacf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:02.230521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a8df00c7-c046-473c-955d-e5aa9eeacf35 None None] MFA Rules not processed for user `c037029a4d494bcab2968cdb7f9a9917`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:02.231717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a8df00c7-c046-473c-955d-e5aa9eeacf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:02.281430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:02.327601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 654/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:02.623079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860a376-ad03-4cec-ae59-4c194c9cb145 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:02.623356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860a376-ad03-4cec-ae59-4c194c9cb145 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:02.623620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860a376-ad03-4cec-ae59-4c194c9cb145 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:02.624998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 663/1317] 10.222.0.33 () {54 vars in 964 bytes} [Mon Oct 3 15:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:02.631108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576975-ff37-447f-8c0b-01695eee6fd3 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:02.631413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576975-ff37-447f-8c0b-01695eee6fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:02.631679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2576975-ff37-447f-8c0b-01695eee6fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:02.705560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b2576975-ff37-447f-8c0b-01695eee6fd3 None None] MFA Rules not processed for user `2d007a3d205345aebcc36a36d039da2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:02.707284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b2576975-ff37-447f-8c0b-01695eee6fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:02.782200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 655/1318] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Oct 3 15:51:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:02.791681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860a376-ad03-4cec-ae59-4c194c9cb145 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:02.793244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b860a376-ad03-4cec-ae59-4c194c9cb145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:02.803776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:02.896074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860a376-ad03-4cec-ae59-4c194c9cb145 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:02.969135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a462813-f8e0-493d-822c-ed535c88d63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:02.969732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a462813-f8e0-493d-822c-ed535c88d63e service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:02.970013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a462813-f8e0-493d-822c-ed535c88d63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:02.970285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a462813-f8e0-493d-822c-ed535c88d63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:02.971872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-4a462813-f8e0-493d-822c-ed535c88d63e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:03.034433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:03.081024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a462813-f8e0-493d-822c-ed535c88d63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:03.083149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a462813-f8e0-493d-822c-ed535c88d63e service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:03.101047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 664/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:06.161473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974c37-c9ec-4898-9bda-f4f9aa5341fe None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:06.162242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974c37-c9ec-4898-9bda-f4f9aa5341fe None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:06.162242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7974c37-c9ec-4898-9bda-f4f9aa5341fe None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:06.163342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 656/1320] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:06.170249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1303-6f50-4315-8a99-2dee662a9280 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:06.170728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1303-6f50-4315-8a99-2dee662a9280 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:06.171082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1303-6f50-4315-8a99-2dee662a9280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:06.236547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-165e1303-6f50-4315-8a99-2dee662a9280 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:06.237807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-165e1303-6f50-4315-8a99-2dee662a9280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:06.334090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 665/1321] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:06.622898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974c37-c9ec-4898-9bda-f4f9aa5341fe None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:06.624449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f7974c37-c9ec-4898-9bda-f4f9aa5341fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:06.674669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:06.728743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7974c37-c9ec-4898-9bda-f4f9aa5341fe None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:06.802431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b80501-32b7-4387-ba6d-9c8ec3b98bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:06.803007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b80501-32b7-4387-ba6d-9c8ec3b98bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:06.803279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b80501-32b7-4387-ba6d-9c8ec3b98bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:06.803543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b80501-32b7-4387-ba6d-9c8ec3b98bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:06.805172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b1b80501-32b7-4387-ba6d-9c8ec3b98bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:06.899068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:06.912483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b80501-32b7-4387-ba6d-9c8ec3b98bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:06.913942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b80501-32b7-4387-ba6d-9c8ec3b98bc6 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:06.931572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 657/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:09.762485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c628337-14e2-4f56-adf3-0ac845e4f762 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:09.762987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c628337-14e2-4f56-adf3-0ac845e4f762 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:09.762987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c628337-14e2-4f56-adf3-0ac845e4f762 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:09.764427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 666/1323] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:09.770311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f6f1-94b2-42ff-90d7-7093a7463286 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:09.770983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f6f1-94b2-42ff-90d7-7093a7463286 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:09.770983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15f6f1-94b2-42ff-90d7-7093a7463286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:09.841601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4e15f6f1-94b2-42ff-90d7-7093a7463286 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:09.843098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4e15f6f1-94b2-42ff-90d7-7093a7463286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:09.948751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 658/1324] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:10.234190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c628337-14e2-4f56-adf3-0ac845e4f762 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:10.235653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8c628337-14e2-4f56-adf3-0ac845e4f762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:10.256507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:10.349859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c628337-14e2-4f56-adf3-0ac845e4f762 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:10.429919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d5179-d797-4050-804c-280f6813d058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:10.430516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d5179-d797-4050-804c-280f6813d058 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:10.430789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d5179-d797-4050-804c-280f6813d058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:10.431053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d5179-d797-4050-804c-280f6813d058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:10.432628 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2f7d5179-d797-4050-804c-280f6813d058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:10.505145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:10.543210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d5179-d797-4050-804c-280f6813d058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:10.544674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d5179-d797-4050-804c-280f6813d058 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:10.563678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 667/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:12.784198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fcec8-9ab6-45c8-8214-6beeb226d7a3 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:12.785424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fcec8-9ab6-45c8-8214-6beeb226d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:12.785617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fcec8-9ab6-45c8-8214-6beeb226d7a3 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:12.787108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 659/1326] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:12.792443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d25e3-303e-4629-8b7f-a165ecebcc24 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:12.792907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d25e3-303e-4629-8b7f-a165ecebcc24 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:12.793027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d25e3-303e-4629-8b7f-a165ecebcc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:12.860410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a02d25e3-303e-4629-8b7f-a165ecebcc24 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:12.861650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a02d25e3-303e-4629-8b7f-a165ecebcc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:12.968776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 668/1327] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:13.252870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fcec8-9ab6-45c8-8214-6beeb226d7a3 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:13.255050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bd4fcec8-9ab6-45c8-8214-6beeb226d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:13.316498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:13.360129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fcec8-9ab6-45c8-8214-6beeb226d7a3 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:13.435592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620193ac-d47d-4890-99c5-5d768862392e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:13.436190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620193ac-d47d-4890-99c5-5d768862392e service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:13.436460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620193ac-d47d-4890-99c5-5d768862392e service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:13.436739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620193ac-d47d-4890-99c5-5d768862392e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:13.438324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-620193ac-d47d-4890-99c5-5d768862392e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:13.544875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:13.547617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620193ac-d47d-4890-99c5-5d768862392e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:13.549778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620193ac-d47d-4890-99c5-5d768862392e service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:13.569813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 660/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:15.870298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27236-a148-4726-bf2d-1af749e53d87 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:15.870819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27236-a148-4726-bf2d-1af749e53d87 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:15.870819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd27236-a148-4726-bf2d-1af749e53d87 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:15.872209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 669/1329] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:15.877423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbcd68-7803-494c-8a19-edebb6e641a7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:15.877832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbcd68-7803-494c-8a19-edebb6e641a7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:15.878018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbcd68-7803-494c-8a19-edebb6e641a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:15.950336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7afbcd68-7803-494c-8a19-edebb6e641a7 None None] MFA Rules not processed for user `c037029a4d494bcab2968cdb7f9a9917`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:15.951528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7afbcd68-7803-494c-8a19-edebb6e641a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:16.046757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 661/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:16.345767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27236-a148-4726-bf2d-1af749e53d87 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:16.347929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8bd27236-a148-4726-bf2d-1af749e53d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:16.379838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:16.457331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd27236-a148-4726-bf2d-1af749e53d87 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:16.533522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbcef1-9cfa-40b8-8d77-ccf766cecda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:16.534205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcef1-9cfa-40b8-8d77-ccf766cecda6 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:16.534550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcef1-9cfa-40b8-8d77-ccf766cecda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:16.534866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbcef1-9cfa-40b8-8d77-ccf766cecda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:16.536533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9adbcef1-9cfa-40b8-8d77-ccf766cecda6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:16.620816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:16.654557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbcef1-9cfa-40b8-8d77-ccf766cecda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:16.656011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbcef1-9cfa-40b8-8d77-ccf766cecda6 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:16.674474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 670/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:19.072197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1682d-8b7a-4364-b712-0b43a3df7f85 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:19.072751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1682d-8b7a-4364-b712-0b43a3df7f85 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:19.072751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1682d-8b7a-4364-b712-0b43a3df7f85 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:19.074096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 662/1332] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:19.079299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d835ad-4b7b-4ddc-93a8-2f822633300e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:19.079688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d835ad-4b7b-4ddc-93a8-2f822633300e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:19.079857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d835ad-4b7b-4ddc-93a8-2f822633300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:19.146716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-39d835ad-4b7b-4ddc-93a8-2f822633300e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:19.147929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-39d835ad-4b7b-4ddc-93a8-2f822633300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:19.245376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 671/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:19.536650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1682d-8b7a-4364-b712-0b43a3df7f85 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:19.538220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8ab1682d-8b7a-4364-b712-0b43a3df7f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:19.606763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:19.640450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1682d-8b7a-4364-b712-0b43a3df7f85 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:19.715137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d757f0-f347-4139-afcd-3755d253ecbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:19.715739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d757f0-f347-4139-afcd-3755d253ecbc service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:19.716012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d757f0-f347-4139-afcd-3755d253ecbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:19.716277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d757f0-f347-4139-afcd-3755d253ecbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:19.718174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d0d757f0-f347-4139-afcd-3755d253ecbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:19.826849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d757f0-f347-4139-afcd-3755d253ecbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:19.828276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d757f0-f347-4139-afcd-3755d253ecbc service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:19.838874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:19.847814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 663/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:22.063941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1400d9-019f-48d1-a477-d296b8334589 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:22.064555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1400d9-019f-48d1-a477-d296b8334589 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:22.064555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1400d9-019f-48d1-a477-d296b8334589 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:22.066915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 672/1335] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:22.072139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abf787-d388-43a7-8c3f-076e7e77ee92 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:22.072592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abf787-d388-43a7-8c3f-076e7e77ee92 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:22.072707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abf787-d388-43a7-8c3f-076e7e77ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:22.140899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f0abf787-d388-43a7-8c3f-076e7e77ee92 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:22.142100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f0abf787-d388-43a7-8c3f-076e7e77ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:22.236158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 664/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:22.526814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1400d9-019f-48d1-a477-d296b8334589 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:22.528343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5d1400d9-019f-48d1-a477-d296b8334589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:22.568649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:22.633008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1400d9-019f-48d1-a477-d296b8334589 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:22.706381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fff47f-05e6-4419-930f-0624bdc9e579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:22.706973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff47f-05e6-4419-930f-0624bdc9e579 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:22.707253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff47f-05e6-4419-930f-0624bdc9e579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:22.707525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fff47f-05e6-4419-930f-0624bdc9e579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:22.709168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d6fff47f-05e6-4419-930f-0624bdc9e579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:22.793149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:22.819494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fff47f-05e6-4419-930f-0624bdc9e579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:22.820917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fff47f-05e6-4419-930f-0624bdc9e579 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:22.839675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 673/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:25.287182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8c585-0d46-43aa-ad84-b5f52bb0f53f None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:25.287750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8c585-0d46-43aa-ad84-b5f52bb0f53f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:25.287750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8c585-0d46-43aa-ad84-b5f52bb0f53f None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:25.289083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 665/1338] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:25.294297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e381c-0c3a-416c-9518-299502895eba None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:25.294724 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e381c-0c3a-416c-9518-299502895eba None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:25.294842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e381c-0c3a-416c-9518-299502895eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:25.365184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-3b1e381c-0c3a-416c-9518-299502895eba None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:25.366378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3b1e381c-0c3a-416c-9518-299502895eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:25.460448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 674/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:51:25.757323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8c585-0d46-43aa-ad84-b5f52bb0f53f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:25.758864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-53e8c585-0d46-43aa-ad84-b5f52bb0f53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:25.835004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:25.863308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8c585-0d46-43aa-ad84-b5f52bb0f53f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:25.935317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b926ff3-5953-4824-99ce-24f502bc9062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:25.935920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b926ff3-5953-4824-99ce-24f502bc9062 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:25.936197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b926ff3-5953-4824-99ce-24f502bc9062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:25.936465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b926ff3-5953-4824-99ce-24f502bc9062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:25.938100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1b926ff3-5953-4824-99ce-24f502bc9062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:26.050364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b926ff3-5953-4824-99ce-24f502bc9062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:26.051724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b926ff3-5953-4824-99ce-24f502bc9062 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:26.069969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 666/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:30.160158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d9127-375b-496b-b5ed-2d631bdcebf5 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:30.160666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d9127-375b-496b-b5ed-2d631bdcebf5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:30.160666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d9127-375b-496b-b5ed-2d631bdcebf5 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:30.162011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 675/1341] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:30.167409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814167-1072-4d45-8bf2-dd0dc48621c8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:30.167824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814167-1072-4d45-8bf2-dd0dc48621c8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:30.168017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee814167-1072-4d45-8bf2-dd0dc48621c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:30.173575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:30.234446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ee814167-1072-4d45-8bf2-dd0dc48621c8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:30.235638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ee814167-1072-4d45-8bf2-dd0dc48621c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:30.334983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 667/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:51:30.617540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d9127-375b-496b-b5ed-2d631bdcebf5 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:30.619058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-726d9127-375b-496b-b5ed-2d631bdcebf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:30.667728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:30.723001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d9127-375b-496b-b5ed-2d631bdcebf5 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:30.794789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cb186-4b07-4707-b7c9-261a5595492a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:30.795388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cb186-4b07-4707-b7c9-261a5595492a service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:30.795661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cb186-4b07-4707-b7c9-261a5595492a service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:30.795926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cb186-4b07-4707-b7c9-261a5595492a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:30.797527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2a4cb186-4b07-4707-b7c9-261a5595492a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:30.889614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:30.906444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cb186-4b07-4707-b7c9-261a5595492a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:30.907904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cb186-4b07-4707-b7c9-261a5595492a service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:30.926590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 676/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:33.745614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a09d2-f09c-4edc-8bc8-c94a33df6a4e None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:33.746186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a09d2-f09c-4edc-8bc8-c94a33df6a4e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:33.746186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a09d2-f09c-4edc-8bc8-c94a33df6a4e None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:33.747573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 668/1344] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:33.753187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf164ad-7775-4165-bbce-b67766b636cf None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:33.753598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf164ad-7775-4165-bbce-b67766b636cf None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:33.753740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf164ad-7775-4165-bbce-b67766b636cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:33.819845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-aaf164ad-7775-4165-bbce-b67766b636cf None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:33.821100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-aaf164ad-7775-4165-bbce-b67766b636cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:33.921226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 677/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:34.208533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a09d2-f09c-4edc-8bc8-c94a33df6a4e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:34.210143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7d2a09d2-f09c-4edc-8bc8-c94a33df6a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:34.308286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:34.328224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a09d2-f09c-4edc-8bc8-c94a33df6a4e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:34.407896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcaef1d-9e36-4201-99c2-e60d26d5bcba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:34.408597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaef1d-9e36-4201-99c2-e60d26d5bcba service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:34.409026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaef1d-9e36-4201-99c2-e60d26d5bcba service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:34.409293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcaef1d-9e36-4201-99c2-e60d26d5bcba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:34.410901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5fcaef1d-9e36-4201-99c2-e60d26d5bcba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:34.556209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcaef1d-9e36-4201-99c2-e60d26d5bcba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:34.558425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcaef1d-9e36-4201-99c2-e60d26d5bcba service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:34.577519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 669/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:38.822025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e131b-b3d3-4050-ad81-620c5406385f None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:38.822515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e131b-b3d3-4050-ad81-620c5406385f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:38.822610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e131b-b3d3-4050-ad81-620c5406385f None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:38.824755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 678/1347] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:38.829523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6806e-1656-465b-9285-8a4ea0dfc7b7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:38.830068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6806e-1656-465b-9285-8a4ea0dfc7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:38.830395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6806e-1656-465b-9285-8a4ea0dfc7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:38.846571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:38.905862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b0e6806e-1656-465b-9285-8a4ea0dfc7b7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:38.907199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b0e6806e-1656-465b-9285-8a4ea0dfc7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:39.018711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 670/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:51:39.305048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e131b-b3d3-4050-ad81-620c5406385f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:39.306576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ab6e131b-b3d3-4050-ad81-620c5406385f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:39.363814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:39.409585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e131b-b3d3-4050-ad81-620c5406385f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:39.483336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26051c38-d043-4845-b86c-1871ed690b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:39.483935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26051c38-d043-4845-b86c-1871ed690b67 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:39.484208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26051c38-d043-4845-b86c-1871ed690b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:39.484473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26051c38-d043-4845-b86c-1871ed690b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:39.486215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-26051c38-d043-4845-b86c-1871ed690b67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:39.592447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:39.597147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26051c38-d043-4845-b86c-1871ed690b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:39.598589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26051c38-d043-4845-b86c-1871ed690b67 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:39.616595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 679/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:44.725406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50450a03-5be6-4d78-bdf5-eda8d633248a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:44.725966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50450a03-5be6-4d78-bdf5-eda8d633248a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:44.726504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50450a03-5be6-4d78-bdf5-eda8d633248a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:44.728028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 671/1350] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 15:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:44.735222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb84fc-5e26-4f71-81c0-9eeb725da71d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:44.738348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb84fc-5e26-4f71-81c0-9eeb725da71d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:44.738348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb84fc-5e26-4f71-81c0-9eeb725da71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:44.822401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-4abb84fc-5e26-4f71-81c0-9eeb725da71d None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:44.823614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-4abb84fc-5e26-4f71-81c0-9eeb725da71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:44.980460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 680/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 15:51:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:49.017698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=59 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:49.213910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a802d9-e1aa-4a80-a421-10355fc24872 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:49.214510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a802d9-e1aa-4a80-a421-10355fc24872 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:49.214893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a802d9-e1aa-4a80-a421-10355fc24872 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:49.216654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 672/1352] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 15:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:49.222244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2aeea-3416-40f3-8e29-762c7efaf7f6 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:49.222718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2aeea-3416-40f3-8e29-762c7efaf7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:49.222823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2aeea-3416-40f3-8e29-762c7efaf7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:49.279537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:49.306769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-00d2aeea-3416-40f3-8e29-762c7efaf7f6 None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:49.308078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-00d2aeea-3416-40f3-8e29-762c7efaf7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:49.429089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 681/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 15:51:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:50.863835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ae4a9-cac5-4737-933c-80936b388acb None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:50.864370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ae4a9-cac5-4737-933c-80936b388acb None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:50.864479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ae4a9-cac5-4737-933c-80936b388acb None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:50.865766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 673/1354] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 15:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:50.870670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68eb3-9a51-4637-9990-2710efe53f83 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:50.871083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68eb3-9a51-4637-9990-2710efe53f83 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:50.871209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae68eb3-9a51-4637-9990-2710efe53f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:50.951247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-eae68eb3-9a51-4637-9990-2710efe53f83 None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:50.952707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-eae68eb3-9a51-4637-9990-2710efe53f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:50.996566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:51.061706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 682/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 15:51:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:54.710354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97062a6-8cb7-445a-b016-bcda6703e6c8 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:54.710920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97062a6-8cb7-445a-b016-bcda6703e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:54.710920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97062a6-8cb7-445a-b016-bcda6703e6c8 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:54.712134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 674/1356] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:51:54.717912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f577-3f21-4d94-ac0b-83ede736fffe None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:54.718347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f577-3f21-4d94-ac0b-83ede736fffe None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:54.718456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2f577-3f21-4d94-ac0b-83ede736fffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:54.792801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-09f2f577-3f21-4d94-ac0b-83ede736fffe None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:54.794356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-09f2f577-3f21-4d94-ac0b-83ede736fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:54.901903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 683/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:51:55.209091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c403f-7204-499b-a312-bed69ee895d8 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:55.209581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c403f-7204-499b-a312-bed69ee895d8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:55.210059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c403f-7204-499b-a312-bed69ee895d8 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:55.211939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 675/1358] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 15:51:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:55.219437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2530f1-17ad-4721-8307-3adf3623c1fe None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:55.219715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2530f1-17ad-4721-8307-3adf3623c1fe None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:55.220547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2530f1-17ad-4721-8307-3adf3623c1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:55.240888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:55.305890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ff2530f1-17ad-4721-8307-3adf3623c1fe None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:55.307282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ff2530f1-17ad-4721-8307-3adf3623c1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:55.434676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 684/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 15:51:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:55.445038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c403f-7204-499b-a312-bed69ee895d8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:55.446664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-607c403f-7204-499b-a312-bed69ee895d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:55.609084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c403f-7204-499b-a312-bed69ee895d8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:55.684955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:55.730038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef82abd-3842-4589-bb8c-c7e093bfa17d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:55.730724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef82abd-3842-4589-bb8c-c7e093bfa17d service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:55.731004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef82abd-3842-4589-bb8c-c7e093bfa17d service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:55.731315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef82abd-3842-4589-bb8c-c7e093bfa17d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:55.733055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cef82abd-3842-4589-bb8c-c7e093bfa17d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:55.851609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef82abd-3842-4589-bb8c-c7e093bfa17d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:55.853578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef82abd-3842-4589-bb8c-c7e093bfa17d service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:55.877969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 676/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:51:58.299853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f9f2-d04a-4f9c-8e63-1118815e3091 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:58.299853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f9f2-d04a-4f9c-8e63-1118815e3091 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:58.300574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6f9f2-d04a-4f9c-8e63-1118815e3091 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:58.302156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 685/1361] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:51:58.307924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de365b-031a-4fe8-8d76-9b91a974b19f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:58.308349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de365b-031a-4fe8-8d76-9b91a974b19f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:58.308484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de365b-031a-4fe8-8d76-9b91a974b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:58.384069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-83de365b-031a-4fe8-8d76-9b91a974b19f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:51:58.385471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-83de365b-031a-4fe8-8d76-9b91a974b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:58.424901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:58.490491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 677/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:51:58.790149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f9f2-d04a-4f9c-8e63-1118815e3091 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:51:58.791747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-00c6f9f2-d04a-4f9c-8e63-1118815e3091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:58.875108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:58.973287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6f9f2-d04a-4f9c-8e63-1118815e3091 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:51:59.102621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbad521-0f7e-4ac3-b6f7-2ace9452008f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:51:59.103233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbad521-0f7e-4ac3-b6f7-2ace9452008f service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:51:59.103512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbad521-0f7e-4ac3-b6f7-2ace9452008f service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:51:59.103782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbad521-0f7e-4ac3-b6f7-2ace9452008f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:51:59.105447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-edbad521-0f7e-4ac3-b6f7-2ace9452008f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:51:59.131105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:51:59.203467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbad521-0f7e-4ac3-b6f7-2ace9452008f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:51:59.205670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbad521-0f7e-4ac3-b6f7-2ace9452008f service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:51:59.225046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 686/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:01.722846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad190b-48f5-4f5b-88c9-6f1d09bd576a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:01.723434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad190b-48f5-4f5b-88c9-6f1d09bd576a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:01.723725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad190b-48f5-4f5b-88c9-6f1d09bd576a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:01.725220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 678/1364] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:01.730308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ddd4a-e595-4277-926a-f46ecd249b12 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:01.730746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ddd4a-e595-4277-926a-f46ecd249b12 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:01.730851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ddd4a-e595-4277-926a-f46ecd249b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:01.800637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ab1ddd4a-e595-4277-926a-f46ecd249b12 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:01.802041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ab1ddd4a-e595-4277-926a-f46ecd249b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:01.904380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:01.908776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 687/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:52:02.208991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0d28a-64ef-415c-8f26-8f699fabdb3b None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:02.209271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0d28a-64ef-415c-8f26-8f699fabdb3b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:02.209545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0d28a-64ef-415c-8f26-8f699fabdb3b None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:02.211122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 679/1366] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 15:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:02.217124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be3b7f-bf59-4f6c-a981-759e2acc66c4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:02.217528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be3b7f-bf59-4f6c-a981-759e2acc66c4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:02.217898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be3b7f-bf59-4f6c-a981-759e2acc66c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:02.304706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-27be3b7f-bf59-4f6c-a981-759e2acc66c4 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:02.306054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-27be3b7f-bf59-4f6c-a981-759e2acc66c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:02.436979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 688/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 15:52:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:02.445256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0d28a-64ef-415c-8f26-8f699fabdb3b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:02.446932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2cc0d28a-64ef-415c-8f26-8f699fabdb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:02.612407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0d28a-64ef-415c-8f26-8f699fabdb3b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:02.639479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:02.734431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa3f2b-a21e-4202-8a48-532db31f8f95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:02.735096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa3f2b-a21e-4202-8a48-532db31f8f95 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:02.735561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa3f2b-a21e-4202-8a48-532db31f8f95 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:02.735947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa3f2b-a21e-4202-8a48-532db31f8f95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:02.737713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f2aa3f2b-a21e-4202-8a48-532db31f8f95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:02.845687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa3f2b-a21e-4202-8a48-532db31f8f95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:02.847444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa3f2b-a21e-4202-8a48-532db31f8f95 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:02.867119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 680/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:02.901977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea2894-9941-43e0-b449-80a243415d59 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:02.902700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea2894-9941-43e0-b449-80a243415d59 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:02.902700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea2894-9941-43e0-b449-80a243415d59 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:02.903889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 689/1369] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:02.910228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7c05-0ec3-4530-ae1d-2eab9abb1c8e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:02.910655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7c05-0ec3-4530-ae1d-2eab9abb1c8e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:02.910936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7c05-0ec3-4530-ae1d-2eab9abb1c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:02.953436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:03.001085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d5bc7c05-0ec3-4530-ae1d-2eab9abb1c8e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:03.003086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d5bc7c05-0ec3-4530-ae1d-2eab9abb1c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:03.127385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 681/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:03.463613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9538154-24e4-440e-9fe3-f2b1750e813f None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:03.463613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9538154-24e4-440e-9fe3-f2b1750e813f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:03.463613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9538154-24e4-440e-9fe3-f2b1750e813f None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:03.465255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 690/1371] 10.222.0.33 () {54 vars in 963 bytes} [Mon Oct 3 15:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:03.469729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bc039-754a-492f-a262-44d05a022e96 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:03.471000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bc039-754a-492f-a262-44d05a022e96 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:03.471297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bc039-754a-492f-a262-44d05a022e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:03.551769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-780bc039-754a-492f-a262-44d05a022e96 None None] MFA Rules not processed for user `4396302927da47a3a80eae4c50fdb266`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:03.553006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-780bc039-754a-492f-a262-44d05a022e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:03.603502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:03.644933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 682/1372] 10.222.0.33 () {58 vars in 1060 bytes} [Mon Oct 3 15:52:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:03.652466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9538154-24e4-440e-9fe3-f2b1750e813f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:03.654237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c9538154-24e4-440e-9fe3-f2b1750e813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:03.709678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:03.768678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9538154-24e4-440e-9fe3-f2b1750e813f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:03.854247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950b560-6611-48d6-85e2-5eead9018912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:03.855268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950b560-6611-48d6-85e2-5eead9018912 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:03.855707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950b560-6611-48d6-85e2-5eead9018912 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:03.856151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950b560-6611-48d6-85e2-5eead9018912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:03.858603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0950b560-6611-48d6-85e2-5eead9018912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:03.998126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:04.020438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950b560-6611-48d6-85e2-5eead9018912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:04.022602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950b560-6611-48d6-85e2-5eead9018912 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:04.052676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 691/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:52:05.137074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366994c5-86ef-41d8-9400-075851a8c438 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:05.137654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366994c5-86ef-41d8-9400-075851a8c438 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:05.137654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366994c5-86ef-41d8-9400-075851a8c438 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:05.138939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 683/1374] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:05.144857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc75f-0a6c-4758-9ab8-179d6e13c77d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:05.145277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc75f-0a6c-4758-9ab8-179d6e13c77d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:05.145424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfc75f-0a6c-4758-9ab8-179d6e13c77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:05.211386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d2cfc75f-0a6c-4758-9ab8-179d6e13c77d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:05.212591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d2cfc75f-0a6c-4758-9ab8-179d6e13c77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:05.317434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 692/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:05.608632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366994c5-86ef-41d8-9400-075851a8c438 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:05.610233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-366994c5-86ef-41d8-9400-075851a8c438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:05.775380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366994c5-86ef-41d8-9400-075851a8c438 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:05.829951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:05.894269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c68f0-01d6-4e63-9a6a-4560958a9ff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:05.894873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c68f0-01d6-4e63-9a6a-4560958a9ff8 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:05.895143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c68f0-01d6-4e63-9a6a-4560958a9ff8 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:05.895405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c68f0-01d6-4e63-9a6a-4560958a9ff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:05.897051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5b0c68f0-01d6-4e63-9a6a-4560958a9ff8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:05.993805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c68f0-01d6-4e63-9a6a-4560958a9ff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:05.995408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c68f0-01d6-4e63-9a6a-4560958a9ff8 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:06.014126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 684/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:08.254183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ad59-50f7-4d8e-a3e3-826a3514c117 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:08.254719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ad59-50f7-4d8e-a3e3-826a3514c117 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:08.254719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ad59-50f7-4d8e-a3e3-826a3514c117 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:08.256237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 693/1377] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:08.261369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663fe05-dcc2-4e2d-b39c-0edabe48df34 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:08.261777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663fe05-dcc2-4e2d-b39c-0edabe48df34 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:08.261943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663fe05-dcc2-4e2d-b39c-0edabe48df34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:08.337185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-3663fe05-dcc2-4e2d-b39c-0edabe48df34 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:08.338375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3663fe05-dcc2-4e2d-b39c-0edabe48df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:08.346653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:08.448836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 685/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:08.738946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ad59-50f7-4d8e-a3e3-826a3514c117 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:08.740748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fac0ad59-50f7-4d8e-a3e3-826a3514c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:08.808788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:08.901004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ad59-50f7-4d8e-a3e3-826a3514c117 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:09.023450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196db9ef-c6d5-45c5-99b0-6f82f54d9370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:09.024200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db9ef-c6d5-45c5-99b0-6f82f54d9370 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:09.024569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db9ef-c6d5-45c5-99b0-6f82f54d9370 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:09.024965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196db9ef-c6d5-45c5-99b0-6f82f54d9370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:09.026780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-196db9ef-c6d5-45c5-99b0-6f82f54d9370 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:09.048772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:09.125808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196db9ef-c6d5-45c5-99b0-6f82f54d9370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:09.127909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196db9ef-c6d5-45c5-99b0-6f82f54d9370 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:09.147778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 694/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:11.378423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc1aed-bd6d-4468-975c-fe1122c4c311 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:11.378981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc1aed-bd6d-4468-975c-fe1122c4c311 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:11.379223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc1aed-bd6d-4468-975c-fe1122c4c311 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:11.384617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 686/1380] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:11.386650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f1d1-4be8-4adf-9693-19023faf11cd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:11.387094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f1d1-4be8-4adf-9693-19023faf11cd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:11.387515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3f1d1-4be8-4adf-9693-19023faf11cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:11.459834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c1b3f1d1-4be8-4adf-9693-19023faf11cd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:11.461202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c1b3f1d1-4be8-4adf-9693-19023faf11cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:11.566072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:11.579960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 695/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:11.878469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc1aed-bd6d-4468-975c-fe1122c4c311 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:11.880841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d7fc1aed-bd6d-4468-975c-fe1122c4c311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:12.050884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:12.066008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc1aed-bd6d-4468-975c-fe1122c4c311 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:12.209570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60f9a3-c958-40d4-b1b3-aceef873bf4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:12.210235 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60f9a3-c958-40d4-b1b3-aceef873bf4f service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:12.210576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60f9a3-c958-40d4-b1b3-aceef873bf4f service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:12.210918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60f9a3-c958-40d4-b1b3-aceef873bf4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:12.212964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2d60f9a3-c958-40d4-b1b3-aceef873bf4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:12.327740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60f9a3-c958-40d4-b1b3-aceef873bf4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:12.329993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60f9a3-c958-40d4-b1b3-aceef873bf4f service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:12.340198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:12.350308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 687/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:15.099795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071072-bf90-43e1-bd41-d1f8266efac9 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:15.100360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071072-bf90-43e1-bd41-d1f8266efac9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:15.100360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071072-bf90-43e1-bd41-d1f8266efac9 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:15.101816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 696/1383] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:15.107810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a024-07cd-4398-abe9-b8c6fbd8a0dd None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:15.108267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a024-07cd-4398-abe9-b8c6fbd8a0dd None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:15.108371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a024-07cd-4398-abe9-b8c6fbd8a0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:15.177595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-fe82a024-07cd-4398-abe9-b8c6fbd8a0dd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:15.178871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fe82a024-07cd-4398-abe9-b8c6fbd8a0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:15.280867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 688/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:15.580917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071072-bf90-43e1-bd41-d1f8266efac9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:15.582414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9a071072-bf90-43e1-bd41-d1f8266efac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:15.756211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071072-bf90-43e1-bd41-d1f8266efac9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:15.830685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:15.884049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9263d2-ff44-4905-b9e5-f996e416c508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:15.884664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9263d2-ff44-4905-b9e5-f996e416c508 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:15.884983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9263d2-ff44-4905-b9e5-f996e416c508 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:15.885260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9263d2-ff44-4905-b9e5-f996e416c508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:15.886925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0e9263d2-ff44-4905-b9e5-f996e416c508 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:15.981918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9263d2-ff44-4905-b9e5-f996e416c508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:15.983614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9263d2-ff44-4905-b9e5-f996e416c508 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:16.004732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 697/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:52:18.270371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a56dd-7e24-4601-a3a9-989cc07d1865 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:18.270940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a56dd-7e24-4601-a3a9-989cc07d1865 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:18.270940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a56dd-7e24-4601-a3a9-989cc07d1865 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:18.272235 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 689/1386] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:18.277587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68087671-f852-4749-98d2-7c2f321e1bae None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:18.278025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68087671-f852-4749-98d2-7c2f321e1bae None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:18.278257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68087671-f852-4749-98d2-7c2f321e1bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:18.348049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-68087671-f852-4749-98d2-7c2f321e1bae None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:18.349288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-68087671-f852-4749-98d2-7c2f321e1bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:18.383571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:18.448695 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 698/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:18.744256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a56dd-7e24-4601-a3a9-989cc07d1865 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:18.745807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cd4a56dd-7e24-4601-a3a9-989cc07d1865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:18.823427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:18.895180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a56dd-7e24-4601-a3a9-989cc07d1865 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:19.125562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fbccc-2ecd-4fdd-a262-dbfecc078b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:19.126720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fbccc-2ecd-4fdd-a262-dbfecc078b62 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:19.126992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fbccc-2ecd-4fdd-a262-dbfecc078b62 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:19.127265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fbccc-2ecd-4fdd-a262-dbfecc078b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:19.128886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5b8fbccc-2ecd-4fdd-a262-dbfecc078b62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:19.167315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:19.220440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fbccc-2ecd-4fdd-a262-dbfecc078b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:19.222112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fbccc-2ecd-4fdd-a262-dbfecc078b62 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:19.243762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 690/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:19.351936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbe306-235d-4ed9-85c5-a8a5059b6dcb None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:19.352534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbe306-235d-4ed9-85c5-a8a5059b6dcb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:19.352534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbe306-235d-4ed9-85c5-a8a5059b6dcb None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:19.353889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 699/1389] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:19.359820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688c28-2988-4267-baac-775c94c25958 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:19.360927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688c28-2988-4267-baac-775c94c25958 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:19.360927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688c28-2988-4267-baac-775c94c25958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:19.435052 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a8688c28-2988-4267-baac-775c94c25958 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:19.436196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a8688c28-2988-4267-baac-775c94c25958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:19.538494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 691/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:52:19.841566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ebece-f4d9-4a8e-a469-ad6a8a156118 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:19.841929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ebece-f4d9-4a8e-a469-ad6a8a156118 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:19.842568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ebece-f4d9-4a8e-a469-ad6a8a156118 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:19.843979 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 700/1391] 10.222.0.33 () {54 vars in 962 bytes} [Mon Oct 3 15:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:19.849985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42330e0d-ae0d-4ade-8428-2515dec7455b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:19.850270 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42330e0d-ae0d-4ade-8428-2515dec7455b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:19.850542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42330e0d-ae0d-4ade-8428-2515dec7455b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:19.867325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:19.929438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-42330e0d-ae0d-4ade-8428-2515dec7455b None None] MFA Rules not processed for user `022516f130ec4ad6a9b7c46b417a5690`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:19.930636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-42330e0d-ae0d-4ade-8428-2515dec7455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:20.007938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 692/1392] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Oct 3 15:52:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:20.017677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ebece-f4d9-4a8e-a469-ad6a8a156118 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:20.019284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-884ebece-f4d9-4a8e-a469-ad6a8a156118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:20.122922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ebece-f4d9-4a8e-a469-ad6a8a156118 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:20.186652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:20.203098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88558d-e713-4ad8-839d-110c3c40ac32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022516f130ec4ad6a9b7c46b417a5690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:20.203706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88558d-e713-4ad8-839d-110c3c40ac32 service glance] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:20.203982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88558d-e713-4ad8-839d-110c3c40ac32 service glance] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:20.204837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88558d-e713-4ad8-839d-110c3c40ac32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:20.206444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2c88558d-e713-4ad8-839d-110c3c40ac32 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:20.434311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88558d-e713-4ad8-839d-110c3c40ac32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:20.435660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88558d-e713-4ad8-839d-110c3c40ac32 service glance] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:20.454020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 701/1393] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:21.206905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cd716-f8de-4105-ac26-9bda6aa198a2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:21.207412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cd716-f8de-4105-ac26-9bda6aa198a2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:21.207522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cd716-f8de-4105-ac26-9bda6aa198a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:21.276218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-803cd716-f8de-4105-ac26-9bda6aa198a2 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:21.277464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-803cd716-f8de-4105-ac26-9bda6aa198a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:21.334788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:21.372357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 693/1394] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 15:52:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:21.414837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83adf7c-46a7-4ceb-9024-81776ed4ff90 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:21.415127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83adf7c-46a7-4ceb-9024-81776ed4ff90 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:21.415400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83adf7c-46a7-4ceb-9024-81776ed4ff90 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:21.416792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 702/1395] 10.222.0.33 () {54 vars in 943 bytes} [Mon Oct 3 15:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:21.423583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df2f3d-04a4-4e7e-a8fd-3013f8ddcce8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:21.423866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df2f3d-04a4-4e7e-a8fd-3013f8ddcce8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:21.424345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df2f3d-04a4-4e7e-a8fd-3013f8ddcce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:21.465044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f36aa-ed34-4bc4-99c7-455d9bcdc2f1 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:21.465619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f36aa-ed34-4bc4-99c7-455d9bcdc2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:21.465619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f36aa-ed34-4bc4-99c7-455d9bcdc2f1 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:21.466910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 703/1396] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:21.472267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:21.472555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:21.472866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:21.503603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f2df2f3d-04a4-4e7e-a8fd-3013f8ddcce8 None None] MFA Rules not processed for user `7701e2653e4b4ca7a842887c0526862f`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:21.504845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f2df2f3d-04a4-4e7e-a8fd-3013f8ddcce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:21.539492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:21.540699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:21.582734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:21.615047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 694/1397] 10.222.0.33 () {58 vars in 1040 bytes} [Mon Oct 3 15:52:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:21.624325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df2f3d-04a4-4e7e-a8fd-3013f8ddcce8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:21.635175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df2f3d-04a4-4e7e-a8fd-3013f8ddcce8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:21.643630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 704/1398] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:52:21.661738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:21.741309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e49cdb-b354-4793-a0d6-f6bd64ec3a11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:21.742317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e49cdb-b354-4793-a0d6-f6bd64ec3a11 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:21.742760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e49cdb-b354-4793-a0d6-f6bd64ec3a11 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:21.743241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e49cdb-b354-4793-a0d6-f6bd64ec3a11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:21.745038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e3e49cdb-b354-4793-a0d6-f6bd64ec3a11 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:21.834429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e49cdb-b354-4793-a0d6-f6bd64ec3a11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:21.836010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e49cdb-b354-4793-a0d6-f6bd64ec3a11 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:21.844046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 695/1399] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 15:52:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 15:52:21.949741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:21.951057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:22.080075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14ec8a-7bc5-4ccb-b8c1-4723d632b379 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:22.169631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:22.239408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff8647-e429-40c3-a9e9-235bd4ce850c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:22.240090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff8647-e429-40c3-a9e9-235bd4ce850c service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:22.240363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff8647-e429-40c3-a9e9-235bd4ce850c service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:22.240628 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff8647-e429-40c3-a9e9-235bd4ce850c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:22.242263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-42ff8647-e429-40c3-a9e9-235bd4ce850c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:22.347123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff8647-e429-40c3-a9e9-235bd4ce850c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:22.348759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff8647-e429-40c3-a9e9-235bd4ce850c service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:22.369469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 705/1400] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:24.619740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32734f65-dee9-43d3-8a9c-956c1ce7c2af None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:24.620359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32734f65-dee9-43d3-8a9c-956c1ce7c2af None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:24.620359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32734f65-dee9-43d3-8a9c-956c1ce7c2af None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:24.622029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 696/1401] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:24.628317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c6b6-c3d5-438e-bef5-22bc990d231d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:24.628317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c6b6-c3d5-438e-bef5-22bc990d231d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:24.628317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c6b6-c3d5-438e-bef5-22bc990d231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:24.697058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-14e4c6b6-c3d5-438e-bef5-22bc990d231d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:24.698274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-14e4c6b6-c3d5-438e-bef5-22bc990d231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:24.714965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:24.812786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 706/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:25.106150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32734f65-dee9-43d3-8a9c-956c1ce7c2af None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:25.107658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-32734f65-dee9-43d3-8a9c-956c1ce7c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:25.175012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:25.273974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32734f65-dee9-43d3-8a9c-956c1ce7c2af None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:25.482864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60012fd-0fb1-4e28-b1c7-15a71b469702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:25.483490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60012fd-0fb1-4e28-b1c7-15a71b469702 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:25.483753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60012fd-0fb1-4e28-b1c7-15a71b469702 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:25.484024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60012fd-0fb1-4e28-b1c7-15a71b469702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:25.485996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a60012fd-0fb1-4e28-b1c7-15a71b469702 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:25.507254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:25.591143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60012fd-0fb1-4e28-b1c7-15a71b469702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:25.593017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60012fd-0fb1-4e28-b1c7-15a71b469702 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:25.611599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 697/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:27.852658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a5e32-d1e3-4449-9a57-9386fb5c8213 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:27.853294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a5e32-d1e3-4449-9a57-9386fb5c8213 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:27.853294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a5e32-d1e3-4449-9a57-9386fb5c8213 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:27.854596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 707/1404] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:27.859927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30812a-4a36-4261-9d95-df799d98a943 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:27.861003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30812a-4a36-4261-9d95-df799d98a943 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:27.861136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30812a-4a36-4261-9d95-df799d98a943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:27.935723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ae30812a-4a36-4261-9d95-df799d98a943 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:27.936971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ae30812a-4a36-4261-9d95-df799d98a943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:28.034929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:28.050021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 698/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:28.333828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a5e32-d1e3-4449-9a57-9386fb5c8213 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:28.335363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-211a5e32-d1e3-4449-9a57-9386fb5c8213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:28.511608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:28.514500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a5e32-d1e3-4449-9a57-9386fb5c8213 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:28.644141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9f0e9-04f1-465a-bc85-339a7f1cfd2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:28.644667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9f0e9-04f1-465a-bc85-339a7f1cfd2b service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:28.644971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9f0e9-04f1-465a-bc85-339a7f1cfd2b service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:28.645236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9f0e9-04f1-465a-bc85-339a7f1cfd2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:28.646833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a8a9f0e9-04f1-465a-bc85-339a7f1cfd2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:28.744953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9f0e9-04f1-465a-bc85-339a7f1cfd2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:28.746584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9f0e9-04f1-465a-bc85-339a7f1cfd2b service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:28.771055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 708/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:30.980946 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6306681-e261-403e-a747-e30b937e8e68 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:30.981482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6306681-e261-403e-a747-e30b937e8e68 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:30.981482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6306681-e261-403e-a747-e30b937e8e68 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:30.983103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 699/1407] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:52:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:30.990613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de141e-ebcc-45bb-91e8-b32db30f2df2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:30.991118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de141e-ebcc-45bb-91e8-b32db30f2df2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:30.991224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de141e-ebcc-45bb-91e8-b32db30f2df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:30.996045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:31.071742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e9de141e-ebcc-45bb-91e8-b32db30f2df2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:31.073102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e9de141e-ebcc-45bb-91e8-b32db30f2df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:31.219515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 709/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:31.545034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6306681-e261-403e-a747-e30b937e8e68 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:31.546561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b6306681-e261-403e-a747-e30b937e8e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:31.710850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6306681-e261-403e-a747-e30b937e8e68 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:31.784563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:31.840640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3114f-bdf8-4b61-bf55-8778f69f89a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:31.841287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3114f-bdf8-4b61-bf55-8778f69f89a7 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:31.841565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3114f-bdf8-4b61-bf55-8778f69f89a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:31.841859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3114f-bdf8-4b61-bf55-8778f69f89a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:31.843746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f1f3114f-bdf8-4b61-bf55-8778f69f89a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:31.948681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3114f-bdf8-4b61-bf55-8778f69f89a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:31.950588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3114f-bdf8-4b61-bf55-8778f69f89a7 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:31.973045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 700/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:41.198670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=64 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:41.968404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=64 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:44.736862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71265f10-464e-4427-8b8e-cfe4e25d6421 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:44.737431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71265f10-464e-4427-8b8e-cfe4e25d6421 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:44.737431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71265f10-464e-4427-8b8e-cfe4e25d6421 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:44.738679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 710/1410] 10.222.0.33 () {54 vars in 918 bytes} [Mon Oct 3 15:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:44.743884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e2b86-950f-4486-87d4-fbc493639a8a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:44.744291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e2b86-950f-4486-87d4-fbc493639a8a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:44.744416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e2b86-950f-4486-87d4-fbc493639a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:44.819732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-008e2b86-950f-4486-87d4-fbc493639a8a None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:44.820958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-008e2b86-950f-4486-87d4-fbc493639a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:44.925558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 701/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 3 15:52:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:52:44.939517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40105c-1fc2-4724-8306-532b9958ba3e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:44.939798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40105c-1fc2-4724-8306-532b9958ba3e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:44.940074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40105c-1fc2-4724-8306-532b9958ba3e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:44.941453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 711/1412] 10.222.0.33 () {54 vars in 918 bytes} [Mon Oct 3 15:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:44.946015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6a1ef-84a5-48d0-96e1-4d79c00fd81e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:44.946297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6a1ef-84a5-48d0-96e1-4d79c00fd81e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:44.946572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6a1ef-84a5-48d0-96e1-4d79c00fd81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:45.021444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:45.023889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-2cb6a1ef-84a5-48d0-96e1-4d79c00fd81e None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:45.025648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2cb6a1ef-84a5-48d0-96e1-4d79c00fd81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:45.139466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 702/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 3 15:52:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:47.727512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b7efd-bee2-482d-b395-706c5c9bc0df None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:47.728050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b7efd-bee2-482d-b395-706c5c9bc0df None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:47.728050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b7efd-bee2-482d-b395-706c5c9bc0df None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:47.729467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:47.734861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac1f4-fd03-4600-a989-75cd1f7e0f56 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:47.735771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac1f4-fd03-4600-a989-75cd1f7e0f56 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:47.735918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ac1f4-fd03-4600-a989-75cd1f7e0f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:47.803468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0b3ac1f4-fd03-4600-a989-75cd1f7e0f56 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:47.804674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0b3ac1f4-fd03-4600-a989-75cd1f7e0f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:47.859235 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:47.903833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:48.195295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b7efd-bee2-482d-b395-706c5c9bc0df None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:48.196908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f65b7efd-bee2-482d-b395-706c5c9bc0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:48.343537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b7efd-bee2-482d-b395-706c5c9bc0df None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:48.417564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:48.457943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e731e6b-7d9e-4c57-8271-124372a6a548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:48.458474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e731e6b-7d9e-4c57-8271-124372a6a548 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:48.458752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e731e6b-7d9e-4c57-8271-124372a6a548 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:48.459016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e731e6b-7d9e-4c57-8271-124372a6a548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:48.460606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7e731e6b-7d9e-4c57-8271-124372a6a548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:48.549630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e731e6b-7d9e-4c57-8271-124372a6a548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:48.551222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e731e6b-7d9e-4c57-8271-124372a6a548 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:48.569194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 713/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:49.708817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42129e48-d91c-4537-a7f7-043f300cf679 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:49.709351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42129e48-d91c-4537-a7f7-043f300cf679 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:49.709351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42129e48-d91c-4537-a7f7-043f300cf679 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:49.710649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:49.717074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ae535-69c4-4df6-b85b-3e53a02eb5cd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:49.717515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ae535-69c4-4df6-b85b-3e53a02eb5cd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:49.717696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ae535-69c4-4df6-b85b-3e53a02eb5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:49.794098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6f3ae535-69c4-4df6-b85b-3e53a02eb5cd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:49.795289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6f3ae535-69c4-4df6-b85b-3e53a02eb5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:49.822491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:49.890647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:50.044296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42129e48-d91c-4537-a7f7-043f300cf679 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:50.045887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-42129e48-d91c-4537-a7f7-043f300cf679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:50.193936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42129e48-d91c-4537-a7f7-043f300cf679 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:50.234736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:50.308895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc43d2a-b797-4912-ac76-fd3f81107f8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:50.309419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc43d2a-b797-4912-ac76-fd3f81107f8d service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:50.309691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc43d2a-b797-4912-ac76-fd3f81107f8d service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:50.309957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc43d2a-b797-4912-ac76-fd3f81107f8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:50.311542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4fc43d2a-b797-4912-ac76-fd3f81107f8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:50.401344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc43d2a-b797-4912-ac76-fd3f81107f8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:50.402967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc43d2a-b797-4912-ac76-fd3f81107f8d service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:50.421487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 705/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:51.553028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d08b0-57fb-4402-966f-b54611905f27 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:51.553561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d08b0-57fb-4402-966f-b54611905f27 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:51.553561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d08b0-57fb-4402-966f-b54611905f27 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:51.554937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:51.561164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2c8b1-7ff6-4fe5-b15f-03beb44ebd42 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:51.561631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2c8b1-7ff6-4fe5-b15f-03beb44ebd42 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:51.561729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2c8b1-7ff6-4fe5-b15f-03beb44ebd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:51.616859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:51.638594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-afb2c8b1-7ff6-4fe5-b15f-03beb44ebd42 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:51.639788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-afb2c8b1-7ff6-4fe5-b15f-03beb44ebd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:51.733974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:52:51.887716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d08b0-57fb-4402-966f-b54611905f27 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:51.889212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-da5d08b0-57fb-4402-966f-b54611905f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:52.038682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d08b0-57fb-4402-966f-b54611905f27 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:52.051010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:52.155408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71dd8d-f77b-4b13-90a2-6230f2c8d82a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:52.155942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71dd8d-f77b-4b13-90a2-6230f2c8d82a service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:52.156220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71dd8d-f77b-4b13-90a2-6230f2c8d82a service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:52.156490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71dd8d-f77b-4b13-90a2-6230f2c8d82a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:52.158221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9b71dd8d-f77b-4b13-90a2-6230f2c8d82a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:52.247699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71dd8d-f77b-4b13-90a2-6230f2c8d82a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:52.249351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71dd8d-f77b-4b13-90a2-6230f2c8d82a service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:52.267793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 716/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:53.371400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b2f05-8d50-4e30-a177-36b0da05d813 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:53.371919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b2f05-8d50-4e30-a177-36b0da05d813 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:53.372018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b2f05-8d50-4e30-a177-36b0da05d813 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:53.373320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:53.380077 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8629303-1aeb-4d49-a082-d709382b2da3 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:53.380536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8629303-1aeb-4d49-a082-d709382b2da3 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:53.380651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8629303-1aeb-4d49-a082-d709382b2da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:53.400701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:53.455908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e8629303-1aeb-4d49-a082-d709382b2da3 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:53.457124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e8629303-1aeb-4d49-a082-d709382b2da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:53.550949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:53.702663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b2f05-8d50-4e30-a177-36b0da05d813 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:53.704679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d89b2f05-8d50-4e30-a177-36b0da05d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:53.814814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:53.851485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b2f05-8d50-4e30-a177-36b0da05d813 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:53.968174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a2b8c-1bf4-4278-ab04-d8f876887121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:53.968685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a2b8c-1bf4-4278-ab04-d8f876887121 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:53.969089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a2b8c-1bf4-4278-ab04-d8f876887121 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:53.969349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a2b8c-1bf4-4278-ab04-d8f876887121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:53.971603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-501a2b8c-1bf4-4278-ab04-d8f876887121 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:54.047208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:54.060625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a2b8c-1bf4-4278-ab04-d8f876887121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:54.062229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a2b8c-1bf4-4278-ab04-d8f876887121 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:54.081448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 708/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:55.226751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edb4f0-d24c-4013-a51a-822e4b10f14d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:55.227305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edb4f0-d24c-4013-a51a-822e4b10f14d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:55.227305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edb4f0-d24c-4013-a51a-822e4b10f14d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:55.228666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:55.234998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9a1f8-6377-4c7d-a025-6697ea6a26e6 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:55.235973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9a1f8-6377-4c7d-a025-6697ea6a26e6 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:55.236237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9a1f8-6377-4c7d-a025-6697ea6a26e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:55.312131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4ff9a1f8-6377-4c7d-a025-6697ea6a26e6 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:55.314098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4ff9a1f8-6377-4c7d-a025-6697ea6a26e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:55.409075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:55.562175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edb4f0-d24c-4013-a51a-822e4b10f14d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:55.563707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-18edb4f0-d24c-4013-a51a-822e4b10f14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:55.646356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:55.708825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edb4f0-d24c-4013-a51a-822e4b10f14d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:55.825129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd36648-85d0-4189-a40d-b489babb91eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:55.825648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd36648-85d0-4189-a40d-b489babb91eb service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:55.825916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd36648-85d0-4189-a40d-b489babb91eb service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:55.826175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd36648-85d0-4189-a40d-b489babb91eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:55.827738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3bd36648-85d0-4189-a40d-b489babb91eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:55.875111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:55.915850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd36648-85d0-4189-a40d-b489babb91eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:55.917470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd36648-85d0-4189-a40d-b489babb91eb service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:55.934678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 719/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:56.995006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88a127-0b7c-4da0-8846-590a81694d53 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:56.995525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88a127-0b7c-4da0-8846-590a81694d53 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:56.995525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88a127-0b7c-4da0-8846-590a81694d53 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:56.996914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:57.003925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bd371-05b5-4034-b8d1-383d86c07263 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:57.004403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bd371-05b5-4034-b8d1-383d86c07263 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:57.004514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38bd371-05b5-4034-b8d1-383d86c07263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:57.082485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-f38bd371-05b5-4034-b8d1-383d86c07263 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:57.083715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f38bd371-05b5-4034-b8d1-383d86c07263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:57.183059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:52:57.340825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88a127-0b7c-4da0-8846-590a81694d53 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:57.342273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5c88a127-0b7c-4da0-8846-590a81694d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:57.392217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:57.489310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88a127-0b7c-4da0-8846-590a81694d53 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:57.599503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273921d-3866-44cd-a404-0fad564c363c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:57.600028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273921d-3866-44cd-a404-0fad564c363c service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:57.600304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273921d-3866-44cd-a404-0fad564c363c service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:57.600571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273921d-3866-44cd-a404-0fad564c363c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:57.602201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d273921d-3866-44cd-a404-0fad564c363c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:57.608992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:57.693448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273921d-3866-44cd-a404-0fad564c363c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:57.695045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273921d-3866-44cd-a404-0fad564c363c service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:57.712649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 711/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:52:58.786029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0ba7-9310-4b64-8a3a-76633d0bd81d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:58.786475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0ba7-9310-4b64-8a3a-76633d0bd81d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:58.786599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c0ba7-9310-4b64-8a3a-76633d0bd81d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:58.787962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:52:58.794252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1b646-1498-47a8-b61b-c2c8a1b14de0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:58.794693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1b646-1498-47a8-b61b-c2c8a1b14de0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:58.794811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1b646-1498-47a8-b61b-c2c8a1b14de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:58.871105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f6a1b646-1498-47a8-b61b-c2c8a1b14de0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:52:58.872300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f6a1b646-1498-47a8-b61b-c2c8a1b14de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:58.938107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:58.969060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:52:59.120022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c0ba7-9310-4b64-8a3a-76633d0bd81d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:52:59.121558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c67c0ba7-9310-4b64-8a3a-76633d0bd81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:59.139386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:59.268262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c0ba7-9310-4b64-8a3a-76633d0bd81d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:52:59.362899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:52:59.388883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86457f4e-345b-4c1b-9746-8913116fd3dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:52:59.389414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86457f4e-345b-4c1b-9746-8913116fd3dd service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:52:59.389690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86457f4e-345b-4c1b-9746-8913116fd3dd service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:52:59.390047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86457f4e-345b-4c1b-9746-8913116fd3dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:52:59.391788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-86457f4e-345b-4c1b-9746-8913116fd3dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:52:59.480849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86457f4e-345b-4c1b-9746-8913116fd3dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:52:59.482459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86457f4e-345b-4c1b-9746-8913116fd3dd service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:52:59.501152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 722/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:00.600338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd49d3-82be-4ee0-abab-7eac01ca0e78 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:00.600907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd49d3-82be-4ee0-abab-7eac01ca0e78 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:00.600907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd49d3-82be-4ee0-abab-7eac01ca0e78 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:00.602224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:00.608460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab768a-2c95-4626-9c11-93f83f511a04 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:00.608960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab768a-2c95-4626-9c11-93f83f511a04 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:00.609069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab768a-2c95-4626-9c11-93f83f511a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:00.685094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a5ab768a-2c95-4626-9c11-93f83f511a04 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:00.686298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a5ab768a-2c95-4626-9c11-93f83f511a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:00.729070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:00.784357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:00.938070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd49d3-82be-4ee0-abab-7eac01ca0e78 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:00.940044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-50fd49d3-82be-4ee0-abab-7eac01ca0e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:01.091056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd49d3-82be-4ee0-abab-7eac01ca0e78 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:01.145148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:01.207418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18390cb1-684a-4add-b7e2-e2ef5c84110d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:01.207938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390cb1-684a-4add-b7e2-e2ef5c84110d service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:01.208215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390cb1-684a-4add-b7e2-e2ef5c84110d service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:01.208479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18390cb1-684a-4add-b7e2-e2ef5c84110d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:01.210098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-18390cb1-684a-4add-b7e2-e2ef5c84110d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:01.301490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18390cb1-684a-4add-b7e2-e2ef5c84110d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:01.303092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18390cb1-684a-4add-b7e2-e2ef5c84110d service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:01.321844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 714/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:02.383672 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372559-d0ad-4d10-aaf5-806550a7369d None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:02.384236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372559-d0ad-4d10-aaf5-806550a7369d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:02.384236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372559-d0ad-4d10-aaf5-806550a7369d None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:02.385647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:02.392805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675db691-9f5d-45aa-9d67-f32705e9d05f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:02.393258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675db691-9f5d-45aa-9d67-f32705e9d05f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:02.393362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675db691-9f5d-45aa-9d67-f32705e9d05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:02.468518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-675db691-9f5d-45aa-9d67-f32705e9d05f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:02.469893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-675db691-9f5d-45aa-9d67-f32705e9d05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:02.472145 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:02.566973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:02.715553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0372559-d0ad-4d10-aaf5-806550a7369d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:02.717140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e0372559-d0ad-4d10-aaf5-806550a7369d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:02.865319 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0372559-d0ad-4d10-aaf5-806550a7369d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:02.892109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:02.976904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d64e19-9be8-43b6-9a8b-8f727d3814cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:02.977515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d64e19-9be8-43b6-9a8b-8f727d3814cb service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:02.977792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d64e19-9be8-43b6-9a8b-8f727d3814cb service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:02.978062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d64e19-9be8-43b6-9a8b-8f727d3814cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:02.979667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-71d64e19-9be8-43b6-9a8b-8f727d3814cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:03.068439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d64e19-9be8-43b6-9a8b-8f727d3814cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:03.070073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d64e19-9be8-43b6-9a8b-8f727d3814cb service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:03.087683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 725/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:05.178728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dce10-b4d0-4e1f-8d78-62c7e4b8c3ec None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:05.179295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dce10-b4d0-4e1f-8d78-62c7e4b8c3ec None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:05.179295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dce10-b4d0-4e1f-8d78-62c7e4b8c3ec None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:05.180605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:05.185859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080f5f6-f65c-4248-a4fe-619c4fed0003 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:05.186277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080f5f6-f65c-4248-a4fe-619c4fed0003 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:05.186427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080f5f6-f65c-4248-a4fe-619c4fed0003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:05.227510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:05.253989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a080f5f6-f65c-4248-a4fe-619c4fed0003 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:05.255207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a080f5f6-f65c-4248-a4fe-619c4fed0003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:05.352807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:05.639286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dce10-b4d0-4e1f-8d78-62c7e4b8c3ec None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:05.640891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-207dce10-b4d0-4e1f-8d78-62c7e4b8c3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:05.745015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dce10-b4d0-4e1f-8d78-62c7e4b8c3ec None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:05.777744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:05.819798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a15f8-d859-460c-8f42-2e7f21742148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:05.820400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a15f8-d859-460c-8f42-2e7f21742148 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:05.820672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a15f8-d859-460c-8f42-2e7f21742148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:05.820970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a15f8-d859-460c-8f42-2e7f21742148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:05.822587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4d3a15f8-d859-460c-8f42-2e7f21742148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:05.930245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a15f8-d859-460c-8f42-2e7f21742148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:05.931590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a15f8-d859-460c-8f42-2e7f21742148 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:05.948987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 717/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:08.264348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c0c86-564c-402b-9ce2-5702e48d185e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:08.264958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c0c86-564c-402b-9ce2-5702e48d185e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:08.264958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c0c86-564c-402b-9ce2-5702e48d185e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:08.266301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:08.271605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3257120-f005-40e7-a5ab-ecfebf1d10b2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:08.272040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3257120-f005-40e7-a5ab-ecfebf1d10b2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:08.272154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3257120-f005-40e7-a5ab-ecfebf1d10b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:08.339681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e3257120-f005-40e7-a5ab-ecfebf1d10b2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:08.340929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e3257120-f005-40e7-a5ab-ecfebf1d10b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:08.354963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:08.439046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:53:08.729096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c0c86-564c-402b-9ce2-5702e48d185e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:08.730703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7d3c0c86-564c-402b-9ce2-5702e48d185e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:08.833643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c0c86-564c-402b-9ce2-5702e48d185e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:08.847370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:08.909261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84555c0b-13ee-4a60-9ccc-a3f4bfbb72de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:08.909860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555c0b-13ee-4a60-9ccc-a3f4bfbb72de service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:08.910133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555c0b-13ee-4a60-9ccc-a3f4bfbb72de service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:08.910398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555c0b-13ee-4a60-9ccc-a3f4bfbb72de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:08.911993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-84555c0b-13ee-4a60-9ccc-a3f4bfbb72de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:09.020768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84555c0b-13ee-4a60-9ccc-a3f4bfbb72de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:09.022188 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84555c0b-13ee-4a60-9ccc-a3f4bfbb72de service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:09.040509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 728/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:11.510410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65de8b-93df-4732-baf3-b4670ffbc934 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:11.510976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65de8b-93df-4732-baf3-b4670ffbc934 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:11.510976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65de8b-93df-4732-baf3-b4670ffbc934 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:11.512282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:11.518516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e0700-019a-46b6-8e88-6bf0babb396d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:11.518934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e0700-019a-46b6-8e88-6bf0babb396d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:11.519068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e0700-019a-46b6-8e88-6bf0babb396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:11.567524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:11.586141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-fc2e0700-019a-46b6-8e88-6bf0babb396d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:11.587292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fc2e0700-019a-46b6-8e88-6bf0babb396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:11.683648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:11.966163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65de8b-93df-4732-baf3-b4670ffbc934 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:11.967679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9a65de8b-93df-4732-baf3-b4670ffbc934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:12.074831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65de8b-93df-4732-baf3-b4670ffbc934 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:12.115798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:12.150771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff67dc9-f383-428b-b2dd-908e86b55859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:12.151378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff67dc9-f383-428b-b2dd-908e86b55859 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:12.151650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff67dc9-f383-428b-b2dd-908e86b55859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:12.151915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff67dc9-f383-428b-b2dd-908e86b55859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:12.153591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bff67dc9-f383-428b-b2dd-908e86b55859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:12.262604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff67dc9-f383-428b-b2dd-908e86b55859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:12.263955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff67dc9-f383-428b-b2dd-908e86b55859 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:12.281424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 720/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:14.548815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457956-05c4-4540-b8ef-c185d281e0e9 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:14.549406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457956-05c4-4540-b8ef-c185d281e0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:14.549406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5457956-05c4-4540-b8ef-c185d281e0e9 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:14.550750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:14.556068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5d900-5b8f-489e-ac2f-b519ad6408fa None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:14.556530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5d900-5b8f-489e-ac2f-b519ad6408fa None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:14.556640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5d900-5b8f-489e-ac2f-b519ad6408fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:14.622664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-53e5d900-5b8f-489e-ac2f-b519ad6408fa None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:14.623788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-53e5d900-5b8f-489e-ac2f-b519ad6408fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:14.644945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:14.720531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:15.012104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5457956-05c4-4540-b8ef-c185d281e0e9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:15.013764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e5457956-05c4-4540-b8ef-c185d281e0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:15.128610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5457956-05c4-4540-b8ef-c185d281e0e9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:15.149378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:15.203694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb071f-5752-49d4-ac8f-9dedad1ecec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:15.204286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb071f-5752-49d4-ac8f-9dedad1ecec9 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:15.204561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb071f-5752-49d4-ac8f-9dedad1ecec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:15.204855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb071f-5752-49d4-ac8f-9dedad1ecec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:15.206476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-05eb071f-5752-49d4-ac8f-9dedad1ecec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:15.315739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb071f-5752-49d4-ac8f-9dedad1ecec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:15.317225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb071f-5752-49d4-ac8f-9dedad1ecec9 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:15.335576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 731/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:17.539606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5563-1b12-4c02-9870-03fa312e96ab None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:17.540175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5563-1b12-4c02-9870-03fa312e96ab None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:17.540175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5563-1b12-4c02-9870-03fa312e96ab None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:17.541516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:17.546795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436141-65fb-4a89-abb5-14afb63a19cd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:17.547200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436141-65fb-4a89-abb5-14afb63a19cd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:17.547955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75436141-65fb-4a89-abb5-14afb63a19cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:17.611807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:17.614916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-75436141-65fb-4a89-abb5-14afb63a19cd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:17.616735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-75436141-65fb-4a89-abb5-14afb63a19cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:17.712820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:17.995830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc5563-1b12-4c02-9870-03fa312e96ab None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:17.997428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-90fc5563-1b12-4c02-9870-03fa312e96ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:18.102028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc5563-1b12-4c02-9870-03fa312e96ab None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:18.154482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:18.178818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef81e46-d436-4431-9ab4-a098d4449c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:18.179416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81e46-d436-4431-9ab4-a098d4449c8b service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:18.179686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81e46-d436-4431-9ab4-a098d4449c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:18.179948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef81e46-d436-4431-9ab4-a098d4449c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:18.181556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1ef81e46-d436-4431-9ab4-a098d4449c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:18.289143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef81e46-d436-4431-9ab4-a098d4449c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:18.290496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef81e46-d436-4431-9ab4-a098d4449c8b service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:18.308453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 723/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:20.642622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87db8-16de-4645-8708-dee2105e13b0 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:20.643152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87db8-16de-4645-8708-dee2105e13b0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:20.643152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e87db8-16de-4645-8708-dee2105e13b0 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:20.644427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:53:20.649703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2378082-123b-445b-8232-bd97050408d4 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:20.650127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2378082-123b-445b-8232-bd97050408d4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:20.650241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2378082-123b-445b-8232-bd97050408d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:20.722143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d2378082-123b-445b-8232-bd97050408d4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:20.723429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d2378082-123b-445b-8232-bd97050408d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:20.752373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:20.822361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:21.106781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e87db8-16de-4645-8708-dee2105e13b0 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:21.108312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-40e87db8-16de-4645-8708-dee2105e13b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:21.212295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e87db8-16de-4645-8708-dee2105e13b0 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:21.240594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:21.286719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5ba8e-a3ec-452c-a196-1386e7c171bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:21.287316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5ba8e-a3ec-452c-a196-1386e7c171bf service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:21.287589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5ba8e-a3ec-452c-a196-1386e7c171bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:21.287855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5ba8e-a3ec-452c-a196-1386e7c171bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:21.289469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-18d5ba8e-a3ec-452c-a196-1386e7c171bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:21.401046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5ba8e-a3ec-452c-a196-1386e7c171bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:21.402497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5ba8e-a3ec-452c-a196-1386e7c171bf service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:21.420378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 734/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:23.616747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084827b-d802-46b6-a704-d764dfc28ae2 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:23.617283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084827b-d802-46b6-a704-d764dfc28ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:23.617283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084827b-d802-46b6-a704-d764dfc28ae2 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:23.618592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:23.623739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794838d-9c46-44eb-9912-d20f5d7d8835 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:23.624144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794838d-9c46-44eb-9912-d20f5d7d8835 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:23.624276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794838d-9c46-44eb-9912-d20f5d7d8835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:23.689833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6794838d-9c46-44eb-9912-d20f5d7d8835 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:23.691018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6794838d-9c46-44eb-9912-d20f5d7d8835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:23.699278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:23.786921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:24.073459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084827b-d802-46b6-a704-d764dfc28ae2 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:24.075011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1084827b-d802-46b6-a704-d764dfc28ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:24.182138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084827b-d802-46b6-a704-d764dfc28ae2 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:24.240159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:24.259261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaff17b-41ee-4c59-ab1b-a6a0d12758f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:24.259863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaff17b-41ee-4c59-ab1b-a6a0d12758f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:24.260134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaff17b-41ee-4c59-ab1b-a6a0d12758f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:24.260399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaff17b-41ee-4c59-ab1b-a6a0d12758f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:24.262017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-caaff17b-41ee-4c59-ab1b-a6a0d12758f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:24.372388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaff17b-41ee-4c59-ab1b-a6a0d12758f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:24.374055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaff17b-41ee-4c59-ab1b-a6a0d12758f4 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:24.393721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 726/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:26.618125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af095a-2c1f-4cc6-ac76-c2e16ccb12eb None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:26.618671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af095a-2c1f-4cc6-ac76-c2e16ccb12eb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:26.618671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af095a-2c1f-4cc6-ac76-c2e16ccb12eb None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:26.620003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:26.625177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f2dc-2f7e-41db-adca-7ed21bf2c4bf None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:26.625623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f2dc-2f7e-41db-adca-7ed21bf2c4bf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:26.625966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f2dc-2f7e-41db-adca-7ed21bf2c4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:26.690502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e5c7f2dc-2f7e-41db-adca-7ed21bf2c4bf None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:26.691696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e5c7f2dc-2f7e-41db-adca-7ed21bf2c4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:26.730265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:26.789072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:27.072970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af095a-2c1f-4cc6-ac76-c2e16ccb12eb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:27.074542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-10af095a-2c1f-4cc6-ac76-c2e16ccb12eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:27.182154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af095a-2c1f-4cc6-ac76-c2e16ccb12eb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:27.220617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:27.255582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de545f-5f49-4571-a78c-e94e6434402e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:27.256176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de545f-5f49-4571-a78c-e94e6434402e service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:27.256448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de545f-5f49-4571-a78c-e94e6434402e service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:27.256707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de545f-5f49-4571-a78c-e94e6434402e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:27.258313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b0de545f-5f49-4571-a78c-e94e6434402e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:27.367070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de545f-5f49-4571-a78c-e94e6434402e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:27.368540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de545f-5f49-4571-a78c-e94e6434402e service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:27.386749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 737/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:29.565976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59827371-09ab-423c-8fcf-8609c3c3530b None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:29.566571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59827371-09ab-423c-8fcf-8609c3c3530b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:29.566571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59827371-09ab-423c-8fcf-8609c3c3530b None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:29.567837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:29.573253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0f4ee-b85d-4120-8a1e-9a8b89880bb5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:29.573724 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0f4ee-b85d-4120-8a1e-9a8b89880bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:29.573724 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0f4ee-b85d-4120-8a1e-9a8b89880bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:29.639175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-60d0f4ee-b85d-4120-8a1e-9a8b89880bb5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:29.640380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-60d0f4ee-b85d-4120-8a1e-9a8b89880bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:29.659136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:29.738356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:30.024418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59827371-09ab-423c-8fcf-8609c3c3530b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:30.026010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-59827371-09ab-423c-8fcf-8609c3c3530b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:30.132978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59827371-09ab-423c-8fcf-8609c3c3530b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:30.199780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:30.206439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e3bb3-cad3-426f-ade3-f1b92c2791cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:30.207038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e3bb3-cad3-426f-ade3-f1b92c2791cb service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:30.207309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e3bb3-cad3-426f-ade3-f1b92c2791cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:30.207570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e3bb3-cad3-426f-ade3-f1b92c2791cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:30.209196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-261e3bb3-cad3-426f-ade3-f1b92c2791cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:30.319523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e3bb3-cad3-426f-ade3-f1b92c2791cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:30.321015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e3bb3-cad3-426f-ade3-f1b92c2791cb service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:30.340163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 729/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:32.539269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513c7a-fe4a-4667-a0fa-15ea9ff92685 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:32.539835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513c7a-fe4a-4667-a0fa-15ea9ff92685 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:32.539835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513c7a-fe4a-4667-a0fa-15ea9ff92685 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:32.541250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:32.546586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398552b-d064-4120-906b-dd96fba5c4c9 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:32.547149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398552b-d064-4120-906b-dd96fba5c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:32.547801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398552b-d064-4120-906b-dd96fba5c4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:32.613411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6398552b-d064-4120-906b-dd96fba5c4c9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:32.614547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6398552b-d064-4120-906b-dd96fba5c4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:32.662708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:32.710640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:32.993787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c513c7a-fe4a-4667-a0fa-15ea9ff92685 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:32.995339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5c513c7a-fe4a-4667-a0fa-15ea9ff92685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:33.097568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c513c7a-fe4a-4667-a0fa-15ea9ff92685 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:33.144598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:33.171186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90e56b-c96a-4fd0-858e-a75e5818eaf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:33.171787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90e56b-c96a-4fd0-858e-a75e5818eaf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:33.172662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90e56b-c96a-4fd0-858e-a75e5818eaf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:33.173102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90e56b-c96a-4fd0-858e-a75e5818eaf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:33.174681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6e90e56b-c96a-4fd0-858e-a75e5818eaf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:33.288990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90e56b-c96a-4fd0-858e-a75e5818eaf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:33.290397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90e56b-c96a-4fd0-858e-a75e5818eaf9 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:33.308613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 740/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:42.709246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=22 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:43.306375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=66 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:53.527842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476483b-d032-4723-9557-5f468d35441c None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:53.528377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476483b-d032-4723-9557-5f468d35441c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:53.528377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476483b-d032-4723-9557-5f468d35441c None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:53.529748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 731/1471] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:53.535107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2cd6d-c745-4eb3-941e-bf31390401d3 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:53.535540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2cd6d-c745-4eb3-941e-bf31390401d3 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:53.535648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2cd6d-c745-4eb3-941e-bf31390401d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:53.602029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-48d2cd6d-c745-4eb3-941e-bf31390401d3 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:53.603526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-48d2cd6d-c745-4eb3-941e-bf31390401d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:53.700846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 741/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:54.000409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e589-dcaf-412d-ba57-bdbc0b83c9e6 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:54.000744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e589-dcaf-412d-ba57-bdbc0b83c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:54.001031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35e589-dcaf-412d-ba57-bdbc0b83c9e6 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:54.002438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 732/1473] 10.222.0.33 () {54 vars in 962 bytes} [Mon Oct 3 15:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:54.007481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0cf1-7e09-4d6a-9983-13234bad1346 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:54.007760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0cf1-7e09-4d6a-9983-13234bad1346 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:54.008035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0cf1-7e09-4d6a-9983-13234bad1346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:54.084531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-0eaf0cf1-7e09-4d6a-9983-13234bad1346 None None] MFA Rules not processed for user `022516f130ec4ad6a9b7c46b417a5690`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:54.085775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0eaf0cf1-7e09-4d6a-9983-13234bad1346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:54.106905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:54.164982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 742/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Oct 3 15:53:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:53:54.173897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e589-dcaf-412d-ba57-bdbc0b83c9e6 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:54.175351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-aa35e589-dcaf-412d-ba57-bdbc0b83c9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:54.285261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35e589-dcaf-412d-ba57-bdbc0b83c9e6 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:54.360439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910902e-c45a-4cae-963e-43b6b46f4d31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022516f130ec4ad6a9b7c46b417a5690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:54.361089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910902e-c45a-4cae-963e-43b6b46f4d31 service glance] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:54.361368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910902e-c45a-4cae-963e-43b6b46f4d31 service glance] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:54.361637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910902e-c45a-4cae-963e-43b6b46f4d31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:54.363266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c910902e-c45a-4cae-963e-43b6b46f4d31 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:54.419132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:54.475296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910902e-c45a-4cae-963e-43b6b46f4d31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:54.476648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910902e-c45a-4cae-963e-43b6b46f4d31 service glance] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:54.495218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 733/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:53:56.788841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba230be-f4fb-4c89-aba8-069d96e83056 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:56.789367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba230be-f4fb-4c89-aba8-069d96e83056 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:56.789367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba230be-f4fb-4c89-aba8-069d96e83056 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:56.790663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 743/1476] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:56.796005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80f7c4-e767-4375-89b7-63e1a722378f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:56.796425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80f7c4-e767-4375-89b7-63e1a722378f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:56.796539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80f7c4-e767-4375-89b7-63e1a722378f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:56.861901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6e80f7c4-e767-4375-89b7-63e1a722378f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:56.863076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6e80f7c4-e767-4375-89b7-63e1a722378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:56.958203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 734/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:53:57.230271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba230be-f4fb-4c89-aba8-069d96e83056 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:53:57.231799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7ba230be-f4fb-4c89-aba8-069d96e83056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:57.358537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba230be-f4fb-4c89-aba8-069d96e83056 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:53:57.409928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:53:57.455615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b295d-4614-45eb-9915-bc1894d1472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:53:57.456216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b295d-4614-45eb-9915-bc1894d1472b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:57.456493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b295d-4614-45eb-9915-bc1894d1472b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:57.456828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b295d-4614-45eb-9915-bc1894d1472b admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:57.457423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b295d-4614-45eb-9915-bc1894d1472b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:53:57.459187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-a90b295d-4614-45eb-9915-bc1894d1472b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:53:57.459187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:53:57.459187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:53:57.460340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b295d-4614-45eb-9915-bc1894d1472b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:53:57.480560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 744/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Oct 3 15:53:57 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:53:59.680297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cf85-c40e-4c6c-a2a8-c7c79f00863d None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:59.680919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cf85-c40e-4c6c-a2a8-c7c79f00863d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:59.680919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cf85-c40e-4c6c-a2a8-c7c79f00863d None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:59.682131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 735/1479] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:53:59.687934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df12b0-08a4-400f-be9c-ffc950e4a1db None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:53:59.688391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df12b0-08a4-400f-be9c-ffc950e4a1db None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:53:59.688500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df12b0-08a4-400f-be9c-ffc950e4a1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:53:59.754297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-49df12b0-08a4-400f-be9c-ffc950e4a1db None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:53:59.755515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-49df12b0-08a4-400f-be9c-ffc950e4a1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:53:59.849815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 745/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:54:00.145564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3cf85-c40e-4c6c-a2a8-c7c79f00863d None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:54:00.147712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-12e3cf85-c40e-4c6c-a2a8-c7c79f00863d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:00.158902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:00.297893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3cf85-c40e-4c6c-a2a8-c7c79f00863d None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:54:00.379792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:00.413661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e617b2-5c0f-405a-bdc6-ca1f56af1301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:54:00.414265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e617b2-5c0f-405a-bdc6-ca1f56af1301 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:00.414536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e617b2-5c0f-405a-bdc6-ca1f56af1301 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:00.414799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e617b2-5c0f-405a-bdc6-ca1f56af1301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:00.416383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-96e617b2-5c0f-405a-bdc6-ca1f56af1301 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:00.507195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e617b2-5c0f-405a-bdc6-ca1f56af1301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:54:00.508909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e617b2-5c0f-405a-bdc6-ca1f56af1301 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:54:00.528033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 736/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:54:02.870042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93f3e-4b7d-40e3-857d-0ca80d6ad792 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:02.870590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93f3e-4b7d-40e3-857d-0ca80d6ad792 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:02.870700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93f3e-4b7d-40e3-857d-0ca80d6ad792 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:02.871999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 746/1482] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:54:02.877452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd2d8b-a2ab-4763-bce8-7a6f3a1e4563 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:02.877889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd2d8b-a2ab-4763-bce8-7a6f3a1e4563 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:02.878015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd2d8b-a2ab-4763-bce8-7a6f3a1e4563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:02.943572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-98dd2d8b-a2ab-4763-bce8-7a6f3a1e4563 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:54:02.944798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-98dd2d8b-a2ab-4763-bce8-7a6f3a1e4563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:02.990875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:03.040652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 737/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:54:03.330482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d93f3e-4b7d-40e3-857d-0ca80d6ad792 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:54:03.332021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e0d93f3e-4b7d-40e3-857d-0ca80d6ad792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:03.356214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:03.436600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d93f3e-4b7d-40e3-857d-0ca80d6ad792 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:54:03.509655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910d70c-342f-4944-b341-508091bdf949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:54:03.510251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d70c-342f-4944-b341-508091bdf949 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:03.510523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d70c-342f-4944-b341-508091bdf949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:03.510787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d70c-342f-4944-b341-508091bdf949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:03.512363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c910d70c-342f-4944-b341-508091bdf949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:03.586700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:03.622170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910d70c-342f-4944-b341-508091bdf949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:54:03.623613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910d70c-342f-4944-b341-508091bdf949 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:54:03.641923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 747/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:54:05.866056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b388ac-97d7-4660-825e-106934642a54 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:05.866635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b388ac-97d7-4660-825e-106934642a54 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:05.866635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b388ac-97d7-4660-825e-106934642a54 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:05.867939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 738/1485] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:54:05.873308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d997b2-bc7a-4139-9f8a-41dfb7d00a2d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:05.873756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d997b2-bc7a-4139-9f8a-41dfb7d00a2d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:05.873868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d997b2-bc7a-4139-9f8a-41dfb7d00a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:05.938635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-f9d997b2-bc7a-4139-9f8a-41dfb7d00a2d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:54:05.939847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f9d997b2-bc7a-4139-9f8a-41dfb7d00a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:06.036403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 748/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:54:06.337273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b388ac-97d7-4660-825e-106934642a54 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:54:06.338817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f6b388ac-97d7-4660-825e-106934642a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:06.443529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b388ac-97d7-4660-825e-106934642a54 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:54:06.519122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:06.521191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6d084-d104-4df5-bb8d-39a27b4e7e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:54:06.522018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6d084-d104-4df5-bb8d-39a27b4e7e8a service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:06.522373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6d084-d104-4df5-bb8d-39a27b4e7e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:06.522851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6d084-d104-4df5-bb8d-39a27b4e7e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:06.524758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5bb6d084-d104-4df5-bb8d-39a27b4e7e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:06.634576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6d084-d104-4df5-bb8d-39a27b4e7e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:54:06.635969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6d084-d104-4df5-bb8d-39a27b4e7e8a service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:54:06.659422 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 739/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:54:08.917724 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c63b-c51f-419c-894e-6e543bcd8fc3 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:08.918257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c63b-c51f-419c-894e-6e543bcd8fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:08.918257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629c63b-c51f-419c-894e-6e543bcd8fc3 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:08.919693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 749/1488] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:54:08.925699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0c25-10d9-4569-83c9-f35fd579eb51 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:08.926132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0c25-10d9-4569-83c9-f35fd579eb51 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:08.926249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0c25-10d9-4569-83c9-f35fd579eb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:08.993026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-9b3e0c25-10d9-4569-83c9-f35fd579eb51 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:54:08.994234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9b3e0c25-10d9-4569-83c9-f35fd579eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:09.046199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:09.090357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 740/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:54:09.383033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c63b-c51f-419c-894e-6e543bcd8fc3 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:54:09.384592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b629c63b-c51f-419c-894e-6e543bcd8fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:09.422917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:09.491636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629c63b-c51f-419c-894e-6e543bcd8fc3 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:54:09.564190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfff26-05b4-404e-a8fd-5f5b034e1543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:54:09.564825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfff26-05b4-404e-a8fd-5f5b034e1543 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:09.565104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfff26-05b4-404e-a8fd-5f5b034e1543 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:09.565374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfff26-05b4-404e-a8fd-5f5b034e1543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:09.566976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-93dfff26-05b4-404e-a8fd-5f5b034e1543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:09.647798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:09.677176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfff26-05b4-404e-a8fd-5f5b034e1543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:54:09.678636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfff26-05b4-404e-a8fd-5f5b034e1543 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:54:09.696392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 750/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:54:12.079174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af4b7a-4618-40ed-97f4-c593bf5220b7 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:12.079748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af4b7a-4618-40ed-97f4-c593bf5220b7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:12.079748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af4b7a-4618-40ed-97f4-c593bf5220b7 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:12.081089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 741/1491] 10.222.0.33 () {54 vars in 927 bytes} [Mon Oct 3 15:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:54:12.086532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87aa6a-c48c-4177-ab27-d27f1ad015fd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:12.086964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87aa6a-c48c-4177-ab27-d27f1ad015fd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:12.087080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87aa6a-c48c-4177-ab27-d27f1ad015fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:12.152776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-aa87aa6a-c48c-4177-ab27-d27f1ad015fd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:54:12.153971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-aa87aa6a-c48c-4177-ab27-d27f1ad015fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:12.260866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 751/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Oct 3 15:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:54:12.548556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af4b7a-4618-40ed-97f4-c593bf5220b7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:54:12.550114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-82af4b7a-4618-40ed-97f4-c593bf5220b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:12.651761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af4b7a-4618-40ed-97f4-c593bf5220b7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:54:12.725210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6706b4e-b99d-416d-bd64-80eafc8062b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:54:12.725811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6706b4e-b99d-416d-bd64-80eafc8062b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:54:12.726088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6706b4e-b99d-416d-bd64-80eafc8062b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:54:12.726357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6706b4e-b99d-416d-bd64-80eafc8062b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:54:12.727952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e6706b4e-b99d-416d-bd64-80eafc8062b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:54:12.737007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:12.836267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6706b4e-b99d-416d-bd64-80eafc8062b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:54:12.837735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6706b4e-b99d-416d-bd64-80eafc8062b0 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:54:12.856545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 742/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:54:22.259262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=82 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:54:22.854199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=50 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:25.674168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:25.674730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:25.674730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:25.753816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:55:25.755012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:25.850374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 752/1494] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:55:25.858815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75547d6e-8c99-4f52-8f38-3c3823afdec1 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:25.859306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75547d6e-8c99-4f52-8f38-3c3823afdec1 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:25.859405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75547d6e-8c99-4f52-8f38-3c3823afdec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:25.938403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-75547d6e-8c99-4f52-8f38-3c3823afdec1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:55:25.939560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-75547d6e-8c99-4f52-8f38-3c3823afdec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:26.035884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 743/1495] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:55:26.040647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:26.041941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:26.054148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a1ee3-f2bc-4a95-a2ed-869ed666a292 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:26.116493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:26.149732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:26.150331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:26.150608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:26.150875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:26.151514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:55:26.151818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:26.152853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:26.152853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:55:26.152853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:55:26.153754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:26.165775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 753/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:55:26.174660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75547d6e-8c99-4f52-8f38-3c3823afdec1 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:26.183847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75547d6e-8c99-4f52-8f38-3c3823afdec1 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:26.247065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:26.282522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:26.283136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:26.283414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:26.283681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:26.284316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:26.286080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING py.warnings [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:26.286080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: warnings.warn(msg) Oct 03 15:55:26.286080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: Oct 03 15:55:26.287277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:26.344344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 744/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:55:26 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:55:26.348823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:26.350372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:26.383409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc901819-79b7-4bb0-9489-cb7cad6034de admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:26.478693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:26.479311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:26.479589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:26.479860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:26.480486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:26.481553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:26.481553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:55:26.481553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:55:26.483386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:26.525085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 754/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:55:26 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:55:26.529667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:26.531218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:26.594281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45939bec-fbc9-45b8-b930-0d32f52a8f00 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:26.679106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:26.690177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:26.690784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:26.691060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:26.691328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:26.691972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:26.693015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING py.warnings [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:26.693015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: warnings.warn(msg) Oct 03 15:55:26.693015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: Oct 03 15:55:26.693905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:26.706915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 745/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:55:26.710912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:26.712429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:26.724908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9029-87c0-4d42-9275-1cea68268719 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:26.749863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:26.820104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16747115-e342-4b6d-97e3-b256563071ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:26.820704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16747115-e342-4b6d-97e3-b256563071ae admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:26.821008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16747115-e342-4b6d-97e3-b256563071ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:26.821274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16747115-e342-4b6d-97e3-b256563071ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:26.821920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16747115-e342-4b6d-97e3-b256563071ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:26.822928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-16747115-e342-4b6d-97e3-b256563071ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:26.822928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:55:26.822928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:55:26.826607 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16747115-e342-4b6d-97e3-b256563071ae admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:26.837306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 755/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:55:26.842081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:26.843686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:26.878059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3033bd0-2ff5-4cd2-abda-75077da8cd2d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:26.973553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785bd65-e3b4-4a28-8839-396ab7b1c111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:26.974488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bd65-e3b4-4a28-8839-396ab7b1c111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:26.974761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bd65-e3b4-4a28-8839-396ab7b1c111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:26.975023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bd65-e3b4-4a28-8839-396ab7b1c111 admin admin] PATH_INFO: `/v3/projects/fa41dab863224f9aaa21e45251d7251c/users/cc2a60573bf445d09de4c8a6d7b8516c/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:26.998618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785bd65-e3b4-4a28-8839-396ab7b1c111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa41dab863224f9aaa21e45251d7251c, user_id=cc2a60573bf445d09de4c8a6d7b8516c, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:26.999634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING py.warnings [None req-9785bd65-e3b4-4a28-8839-396ab7b1c111 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:26.999634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: warnings.warn(msg) Oct 03 15:55:26.999634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: Oct 03 15:55:27.001599 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785bd65-e3b4-4a28-8839-396ab7b1c111 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:27.025601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 746/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:55:26 2022] PUT /identity/v3/projects/fa41dab863224f9aaa21e45251d7251c/users/cc2a60573bf445d09de4c8a6d7b8516c/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:55:27.037966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338032d3-7351-4755-a252-28ec25ff9c57 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:27.038252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338032d3-7351-4755-a252-28ec25ff9c57 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:27.038521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338032d3-7351-4755-a252-28ec25ff9c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:27.095835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-338032d3-7351-4755-a252-28ec25ff9c57 None None] MFA Rules not processed for user `cc2a60573bf445d09de4c8a6d7b8516c`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:55:27.097002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-338032d3-7351-4755-a252-28ec25ff9c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:27.177333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 756/1502] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:55:27.191267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eafe44-1ae8-428c-af3e-a3977301e5d0 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:27.191571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eafe44-1ae8-428c-af3e-a3977301e5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:27.192179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eafe44-1ae8-428c-af3e-a3977301e5d0 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:27.193541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 747/1503] 10.222.0.33 () {54 vars in 963 bytes} [Mon Oct 3 15:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:55:27.198350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ea79d-b625-4fd0-884a-529a6ae311db None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:27.198631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ea79d-b625-4fd0-884a-529a6ae311db None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:27.198897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ea79d-b625-4fd0-884a-529a6ae311db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:27.231003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:27.276175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5c5ea79d-b625-4fd0-884a-529a6ae311db None None] MFA Rules not processed for user `4396302927da47a3a80eae4c50fdb266`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:55:27.277467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5c5ea79d-b625-4fd0-884a-529a6ae311db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:27.360101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 757/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Mon Oct 3 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:55:27.367707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eafe44-1ae8-428c-af3e-a3977301e5d0 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:27.369264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-91eafe44-1ae8-428c-af3e-a3977301e5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:27.420303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:27.474942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eafe44-1ae8-428c-af3e-a3977301e5d0 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:27.549991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:27.550616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:27.550921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:27.551187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:27.552858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:27.654374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:27.655719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:27.659156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:27.674659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 748/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:55:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:55:27.721390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1357b5-f0f0-405c-8bd5-353d1df77433 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:27.721817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1357b5-f0f0-405c-8bd5-353d1df77433 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:27.721938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1357b5-f0f0-405c-8bd5-353d1df77433 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:27.723208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 758/1506] 10.222.0.33 () {52 vars in 830 bytes} [Mon Oct 3 15:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:55:27.726820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:27.728009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:27.826081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6985f-9ffe-44ce-88f5-052ba209610c service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:27.918037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:27.918677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:27.918848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:27.919116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] PATH_INFO: `/v3/users/cc2a60573bf445d09de4c8a6d7b8516c` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:27.932367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2a60573bf445d09de4c8a6d7b8516c)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:27.933417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING py.warnings [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:27.933417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: warnings.warn(msg) Oct 03 15:55:27.933417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: Oct 03 15:55:27.934620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:27.944446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:28.001463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2a60573bf445d09de4c8a6d7b8516c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:28.010444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2a60573bf445d09de4c8a6d7b8516c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:28.033964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2a60573bf445d09de4c8a6d7b8516c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:28.048482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2a60573bf445d09de4c8a6d7b8516c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:28.048961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2a60573bf445d09de4c8a6d7b8516c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:28.054777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 749/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:55:27 2022] DELETE /identity/v3/users/cc2a60573bf445d09de4c8a6d7b8516c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:55:28.059139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:28.059418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:28.059689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:28.116784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:55:28.117984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:28.192875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:28.207050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 759/1508] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:55:28.217554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:28.218766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:28.318317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197d23-3d23-4b3e-adf5-e40f03930596 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:28.376170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:28.396646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e42937-93cb-4549-aeac-24118ee03cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:28.397281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e42937-93cb-4549-aeac-24118ee03cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:28.397552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e42937-93cb-4549-aeac-24118ee03cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:28.397815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e42937-93cb-4549-aeac-24118ee03cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:28.399389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-01e42937-93cb-4549-aeac-24118ee03cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:28.506967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e42937-93cb-4549-aeac-24118ee03cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:28.508323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e42937-93cb-4549-aeac-24118ee03cb2 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:28.527238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 750/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:55:28.881388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:55:28.882625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:55:28.897136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708955ad-6540-4fe0-83f9-4ecf738ab4e5 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:55:29.009125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:55:29.009674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:55:29.009943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:55:29.010204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] PATH_INFO: `/v3/projects/fa41dab863224f9aaa21e45251d7251c` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:55:29.019936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa41dab863224f9aaa21e45251d7251c)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:55:29.020991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:55:29.020991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:55:29.020991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:55:29.022192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:55:29.080222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa41dab863224f9aaa21e45251d7251c', 'request_id': 'req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:29.089919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa41dab863224f9aaa21e45251d7251c', 'request_id': 'req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:29.096678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa41dab863224f9aaa21e45251d7251c', 'request_id': 'req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:29.126692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] The token cache is being invalidate because project fa41dab863224f9aaa21e45251d7251c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:55:29.126974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:29.127390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-174fea8f-9c31-4a2c-b42b-8798b24e3b7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:55:29.128839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 760/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:55:28 2022] DELETE /identity/v3/projects/fa41dab863224f9aaa21e45251d7251c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:55:38.525000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=62 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:55:39.124703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:57:00.418400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a4aa8-b075-48d1-aa7c-74247d989822 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:00.419517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a4aa8-b075-48d1-aa7c-74247d989822 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:00.419517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a4aa8-b075-48d1-aa7c-74247d989822 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:00.421026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:00.425970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a3a7f-f506-4dfa-b6e1-7c74477c6c0d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:00.426426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a3a7f-f506-4dfa-b6e1-7c74477c6c0d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:00.426538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a3a7f-f506-4dfa-b6e1-7c74477c6c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:00.508584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cc6a3a7f-f506-4dfa-b6e1-7c74477c6c0d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:00.509813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cc6a3a7f-f506-4dfa-b6e1-7c74477c6c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:00.614080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:00.769696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a4aa8-b075-48d1-aa7c-74247d989822 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:00.771818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3e7a4aa8-b075-48d1-aa7c-74247d989822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:00.887544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a4aa8-b075-48d1-aa7c-74247d989822 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:00.893158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83035b-a7a6-4908-8cdb-661746ccdce7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:00.893788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83035b-a7a6-4908-8cdb-661746ccdce7 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:00.894088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83035b-a7a6-4908-8cdb-661746ccdce7 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:00.894360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83035b-a7a6-4908-8cdb-661746ccdce7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:00.902706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83035b-a7a6-4908-8cdb-661746ccdce7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:00.904879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83035b-a7a6-4908-8cdb-661746ccdce7 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:00.914304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 752/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:02.016764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa573ca-4183-4a0a-be32-303bb86d441e None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:02.017312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa573ca-4183-4a0a-be32-303bb86d441e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:02.017312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa573ca-4183-4a0a-be32-303bb86d441e None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:02.018679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:02.025255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe58fa-05c0-4dc8-b416-39a8800e8d27 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:02.025716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe58fa-05c0-4dc8-b416-39a8800e8d27 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:02.025835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe58fa-05c0-4dc8-b416-39a8800e8d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:02.047972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-71fe58fa-05c0-4dc8-b416-39a8800e8d27 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:02.049213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-71fe58fa-05c0-4dc8-b416-39a8800e8d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:02.057616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:02.209978 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa573ca-4183-4a0a-be32-303bb86d441e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:02.224490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa573ca-4183-4a0a-be32-303bb86d441e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:02.229908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b470f-2115-42ff-96f3-b4fed93fdf63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:02.230501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b470f-2115-42ff-96f3-b4fed93fdf63 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:02.230780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b470f-2115-42ff-96f3-b4fed93fdf63 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:02.231055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b470f-2115-42ff-96f3-b4fed93fdf63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:02.239100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b470f-2115-42ff-96f3-b4fed93fdf63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:02.240700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b470f-2115-42ff-96f3-b4fed93fdf63 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:02.250734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 763/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:02.545799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028f0bd-81cf-40a4-81b0-402993723ead None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:02.546007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028f0bd-81cf-40a4-81b0-402993723ead None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:02.546253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028f0bd-81cf-40a4-81b0-402993723ead None None] PATH_INFO: `/v3` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:02.547515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 754/1517] 10.222.0.48 () {54 vars in 925 bytes} [Mon Oct 3 15:57:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:57:02.569648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19c438-cd63-4a19-be7d-e6091721e61b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:02.569930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19c438-cd63-4a19-be7d-e6091721e61b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:02.570200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19c438-cd63-4a19-be7d-e6091721e61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:02.630167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ca19c438-cd63-4a19-be7d-e6091721e61b None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:02.631377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ca19c438-cd63-4a19-be7d-e6091721e61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:02.648510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:57:02.704602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 764/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Oct 3 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:06.314960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03fd59-2f81-4b33-9ed0-7782e56817b6 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:06.315590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03fd59-2f81-4b33-9ed0-7782e56817b6 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:06.315691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03fd59-2f81-4b33-9ed0-7782e56817b6 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:06.317352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 755/1519] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 15:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:57:06.322558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37706c46-6883-47ed-8cd3-62dc2fb0a1c0 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:06.323111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37706c46-6883-47ed-8cd3-62dc2fb0a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:06.323111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37706c46-6883-47ed-8cd3-62dc2fb0a1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:06.342644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-37706c46-6883-47ed-8cd3-62dc2fb0a1c0 None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:06.343865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-37706c46-6883-47ed-8cd3-62dc2fb0a1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:06.354028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 765/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 15:57:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:06.362761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03fd59-2f81-4b33-9ed0-7782e56817b6 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:06.379347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03fd59-2f81-4b33-9ed0-7782e56817b6 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:06.384548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d679d8-d721-4ecc-ba56-0414fbc471ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:06.385180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d679d8-d721-4ecc-ba56-0414fbc471ad service placement] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:06.385454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d679d8-d721-4ecc-ba56-0414fbc471ad service placement] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:06.385718 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d679d8-d721-4ecc-ba56-0414fbc471ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:06.394065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d679d8-d721-4ecc-ba56-0414fbc471ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:06.395632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d679d8-d721-4ecc-ba56-0414fbc471ad service placement] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:06.405831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 756/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:57:13.736319 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7bad-b0d4-4e55-954f-a8c71b85caf3 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:13.736866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7bad-b0d4-4e55-954f-a8c71b85caf3 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:13.736866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7bad-b0d4-4e55-954f-a8c71b85caf3 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:13.738146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:13.744232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54699e-68b8-44c5-8f7c-861309c15c88 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:13.744662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54699e-68b8-44c5-8f7c-861309c15c88 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:13.744792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54699e-68b8-44c5-8f7c-861309c15c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:13.764931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-5b54699e-68b8-44c5-8f7c-861309c15c88 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:13.766133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5b54699e-68b8-44c5-8f7c-861309c15c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:13.775289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:57:13.929965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7bad-b0d4-4e55-954f-a8c71b85caf3 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:13.944996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7bad-b0d4-4e55-954f-a8c71b85caf3 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:13.950326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe81c3-40b4-4f12-84a2-1a3894d569a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:13.950914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe81c3-40b4-4f12-84a2-1a3894d569a4 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:13.951193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe81c3-40b4-4f12-84a2-1a3894d569a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:13.951462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe81c3-40b4-4f12-84a2-1a3894d569a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:13.959338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe81c3-40b4-4f12-84a2-1a3894d569a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:13.960935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe81c3-40b4-4f12-84a2-1a3894d569a4 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:13.970424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 767/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:15.037899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e638-6611-4fd5-b972-d9e5cbe4b78d None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:15.038464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e638-6611-4fd5-b972-d9e5cbe4b78d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:15.038464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e638-6611-4fd5-b972-d9e5cbe4b78d None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:15.039803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 15:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:15.046415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcac069-0c49-4fb9-815f-df2dd9b1da8e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:15.046870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcac069-0c49-4fb9-815f-df2dd9b1da8e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:15.046989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcac069-0c49-4fb9-815f-df2dd9b1da8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:15.066674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-8dcac069-0c49-4fb9-815f-df2dd9b1da8e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:15.067865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8dcac069-0c49-4fb9-815f-df2dd9b1da8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:15.075699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 15:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:15.234324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e638-6611-4fd5-b972-d9e5cbe4b78d None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:15.249202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e638-6611-4fd5-b972-d9e5cbe4b78d None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:15.254748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab14243-62dc-4820-bdbe-e7ae0a98e509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:15.255331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab14243-62dc-4820-bdbe-e7ae0a98e509 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:15.255602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab14243-62dc-4820-bdbe-e7ae0a98e509 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:15.255871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab14243-62dc-4820-bdbe-e7ae0a98e509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:15.263874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab14243-62dc-4820-bdbe-e7ae0a98e509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:15.265480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab14243-62dc-4820-bdbe-e7ae0a98e509 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:15.274657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 759/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:19.353620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d47a0-e4fb-4f8b-bd8b-e16b2e991ca2 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:19.354091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d47a0-e4fb-4f8b-bd8b-e16b2e991ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:19.354205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d47a0-e4fb-4f8b-bd8b-e16b2e991ca2 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:19.355510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:19.361245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093ef97-ea27-45a2-a963-6815144d0415 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:19.361685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093ef97-ea27-45a2-a963-6815144d0415 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:19.361781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093ef97-ea27-45a2-a963-6815144d0415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:19.381041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b093ef97-ea27-45a2-a963-6815144d0415 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:19.382234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b093ef97-ea27-45a2-a963-6815144d0415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:19.390206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:57:19.682386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d47a0-e4fb-4f8b-bd8b-e16b2e991ca2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:19.684102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-900d47a0-e4fb-4f8b-bd8b-e16b2e991ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:19.762868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d47a0-e4fb-4f8b-bd8b-e16b2e991ca2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:19.767185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84671ea0-53f9-488d-a0d8-aa23a6fe5346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:19.767784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84671ea0-53f9-488d-a0d8-aa23a6fe5346 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:19.768064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84671ea0-53f9-488d-a0d8-aa23a6fe5346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:19.768336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84671ea0-53f9-488d-a0d8-aa23a6fe5346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:19.778498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84671ea0-53f9-488d-a0d8-aa23a6fe5346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:19.781128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84671ea0-53f9-488d-a0d8-aa23a6fe5346 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:19.790485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 770/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:21.984709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddbce6-9cc4-4b77-8582-0e08c4681092 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:21.985309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddbce6-9cc4-4b77-8582-0e08c4681092 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:21.985423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddbce6-9cc4-4b77-8582-0e08c4681092 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:21.986667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:21.991753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76285117-fd1a-4d15-8aec-5976ae93bd00 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:21.992151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76285117-fd1a-4d15-8aec-5976ae93bd00 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:21.992294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76285117-fd1a-4d15-8aec-5976ae93bd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:22.011439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-76285117-fd1a-4d15-8aec-5976ae93bd00 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:22.012619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-76285117-fd1a-4d15-8aec-5976ae93bd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:22.021272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:22.334884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddbce6-9cc4-4b77-8582-0e08c4681092 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:22.336711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-15ddbce6-9cc4-4b77-8582-0e08c4681092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:22.352452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddbce6-9cc4-4b77-8582-0e08c4681092 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:22.355975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b755b-ec3f-41ce-81a1-dc26b2ca8224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:22.356564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b755b-ec3f-41ce-81a1-dc26b2ca8224 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:22.356872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b755b-ec3f-41ce-81a1-dc26b2ca8224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:22.357141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b755b-ec3f-41ce-81a1-dc26b2ca8224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:22.366704 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b755b-ec3f-41ce-81a1-dc26b2ca8224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:22.368116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b755b-ec3f-41ce-81a1-dc26b2ca8224 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:22.377252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 762/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:26.019075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c50618-b8a2-4fd8-a37a-b7f69c327c55 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:26.019592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c50618-b8a2-4fd8-a37a-b7f69c327c55 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:26.019592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c50618-b8a2-4fd8-a37a-b7f69c327c55 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:26.020904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:26.026127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c259f-b148-4293-adc3-0534a42766bf None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:26.026529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c259f-b148-4293-adc3-0534a42766bf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:26.026698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c259f-b148-4293-adc3-0534a42766bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:26.047247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-cd3c259f-b148-4293-adc3-0534a42766bf None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:26.048447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cd3c259f-b148-4293-adc3-0534a42766bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:26.056367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:26.357844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c50618-b8a2-4fd8-a37a-b7f69c327c55 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:26.359519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-78c50618-b8a2-4fd8-a37a-b7f69c327c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:26.423894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:57:26.438352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c50618-b8a2-4fd8-a37a-b7f69c327c55 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:26.441954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531b98d-b290-4625-b511-affc6f91438c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:26.442573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531b98d-b290-4625-b511-affc6f91438c service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:26.442849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531b98d-b290-4625-b511-affc6f91438c service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:26.443118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531b98d-b290-4625-b511-affc6f91438c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:26.452544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531b98d-b290-4625-b511-affc6f91438c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:26.454023 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531b98d-b290-4625-b511-affc6f91438c service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:26.463435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 773/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:28.644614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bba3e0-ef98-4a20-ab9e-ae7cee6d455a None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:28.645151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bba3e0-ef98-4a20-ab9e-ae7cee6d455a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:28.645151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bba3e0-ef98-4a20-ab9e-ae7cee6d455a None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:28.646505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 15:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:57:28.651799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7e0c-59ad-4472-836f-703c25a1f7c5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:28.652229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7e0c-59ad-4472-836f-703c25a1f7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:28.652332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7e0c-59ad-4472-836f-703c25a1f7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:28.671813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-077f7e0c-59ad-4472-836f-703c25a1f7c5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:57:28.673016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-077f7e0c-59ad-4472-836f-703c25a1f7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:28.681003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 15:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:57:28.983399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bba3e0-ef98-4a20-ab9e-ae7cee6d455a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:57:28.985118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-43bba3e0-ef98-4a20-ab9e-ae7cee6d455a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:57:28.992758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=84 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:57:29.000065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bba3e0-ef98-4a20-ab9e-ae7cee6d455a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:57:29.003587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25765-115c-446b-a1c2-3560f2c5ef3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:57:29.004183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25765-115c-446b-a1c2-3560f2c5ef3c service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:57:29.004461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25765-115c-446b-a1c2-3560f2c5ef3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:57:29.004742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25765-115c-446b-a1c2-3560f2c5ef3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:57:29.013756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25765-115c-446b-a1c2-3560f2c5ef3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:57:29.015295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25765-115c-446b-a1c2-3560f2c5ef3c service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:57:29.024531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 765/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:57:38.667126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=11 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:57:39.020485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:55.054500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6c404-bd36-4df1-98bf-d58f489cd50e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.055585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6c404-bd36-4df1-98bf-d58f489cd50e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.056088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6c404-bd36-4df1-98bf-d58f489cd50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.064050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a172172-9fec-47ab-b2fc-3e26f622002b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.066020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a172172-9fec-47ab-b2fc-3e26f622002b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.066527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a172172-9fec-47ab-b2fc-3e26f622002b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.092841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-faf6c404-bd36-4df1-98bf-d58f489cd50e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.093746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-5a172172-9fec-47ab-b2fc-3e26f622002b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.094951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5a172172-9fec-47ab-b2fc-3e26f622002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.095236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-faf6c404-bd36-4df1-98bf-d58f489cd50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.107296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 766/1540] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:58:55.108593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fd58d-bc45-469c-ae8c-345f37b6c391 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.109009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fd58d-bc45-469c-ae8c-345f37b6c391 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.109319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fd58d-bc45-469c-ae8c-345f37b6c391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.113009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 775/1541] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.114519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf37fd3-b16a-4659-b88b-c6910dbd6bdb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.115018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf37fd3-b16a-4659-b88b-c6910dbd6bdb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.115597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf37fd3-b16a-4659-b88b-c6910dbd6bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.144269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f31fd58d-bc45-469c-ae8c-345f37b6c391 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.145507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f31fd58d-bc45-469c-ae8c-345f37b6c391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.153831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cdf37fd3-b16a-4659-b88b-c6910dbd6bdb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.155557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cdf37fd3-b16a-4659-b88b-c6910dbd6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.157787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 767/1542] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.163939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467804b-e186-4823-a06d-1e71e54c00b8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.164211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467804b-e186-4823-a06d-1e71e54c00b8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.164543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467804b-e186-4823-a06d-1e71e54c00b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.187987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 776/1543] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.190316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e381d-53e7-4b30-ae63-446be46de110 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.190931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e381d-53e7-4b30-ae63-446be46de110 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.191432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e381d-53e7-4b30-ae63-446be46de110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.200753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6467804b-e186-4823-a06d-1e71e54c00b8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.200753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6467804b-e186-4823-a06d-1e71e54c00b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.204970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 768/1544] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.206399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3bcf-4f23-42ff-bdb9-06085be07172 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.206693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3bcf-4f23-42ff-bdb9-06085be07172 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.206974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3bcf-4f23-42ff-bdb9-06085be07172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.229098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-fa6b3bcf-4f23-42ff-bdb9-06085be07172 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.230306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fa6b3bcf-4f23-42ff-bdb9-06085be07172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.235922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-992e381d-53e7-4b30-ae63-446be46de110 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.238102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-992e381d-53e7-4b30-ae63-446be46de110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.239566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 769/1545] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.241923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b3bcf-4f23-42ff-bdb9-06085be07172 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.251488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b3bcf-4f23-42ff-bdb9-06085be07172 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.252508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 777/1546] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.253525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e381d-53e7-4b30-ae63-446be46de110 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.257993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.258894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.259171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.260059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.260714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.261315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.262344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING py.warnings [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:58:55.262344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: warnings.warn(msg) Oct 03 15:58:55.262344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: Oct 03 15:58:55.263243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd431b6-a111-4d26-8cb5-b2ce894b1331 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.263415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e381d-53e7-4b30-ae63-446be46de110 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.269123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c2ed4-bf72-41eb-8762-6f4025955c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.269807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c2ed4-bf72-41eb-8762-6f4025955c04 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.270152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c2ed4-bf72-41eb-8762-6f4025955c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.270488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c2ed4-bf72-41eb-8762-6f4025955c04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.271200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c2ed4-bf72-41eb-8762-6f4025955c04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.271566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c2ed4-bf72-41eb-8762-6f4025955c04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.273280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c2ed4-bf72-41eb-8762-6f4025955c04 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.278088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 770/1547] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.279440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f45c9-1bf2-4c81-918f-7fbaadc237f2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.279717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f45c9-1bf2-4c81-918f-7fbaadc237f2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.279982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f45c9-1bf2-4c81-918f-7fbaadc237f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.289182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 778/1548] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.290740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec95702-46ad-4532-aa97-b97a3e61c2cf None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.291021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec95702-46ad-4532-aa97-b97a3e61c2cf None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.291288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec95702-46ad-4532-aa97-b97a3e61c2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.302711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-640f45c9-1bf2-4c81-918f-7fbaadc237f2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.304050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-640f45c9-1bf2-4c81-918f-7fbaadc237f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.312407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 771/1549] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.312600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1ec95702-46ad-4532-aa97-b97a3e61c2cf None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.316752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb608771-5918-4b26-b6a6-4147512703a3 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.316923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1ec95702-46ad-4532-aa97-b97a3e61c2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.317009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb608771-5918-4b26-b6a6-4147512703a3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.317009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb608771-5918-4b26-b6a6-4147512703a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.324006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 779/1550] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.324904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec95702-46ad-4532-aa97-b97a3e61c2cf None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.334615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bb608771-5918-4b26-b6a6-4147512703a3 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.335498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec95702-46ad-4532-aa97-b97a3e61c2cf None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.335793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bb608771-5918-4b26-b6a6-4147512703a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.340336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.340955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.341238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.341508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.342150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.342452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.344044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 772/1551] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.344338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.345214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f29-eb73-4a2d-a267-3c1aaf898fd4 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.346023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f29-eb73-4a2d-a267-3c1aaf898fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.346291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38131f29-eb73-4a2d-a267-3c1aaf898fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.365019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 780/1552] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.365019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.366622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-38131f29-eb73-4a2d-a267-3c1aaf898fd4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.367823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-38131f29-eb73-4a2d-a267-3c1aaf898fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.370437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4eb9ce-4e62-4b5b-8349-76e05062aea3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.374994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17eaa6-2fc8-4679-8ab8-0737a2640b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.375579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17eaa6-2fc8-4679-8ab8-0737a2640b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.375863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17eaa6-2fc8-4679-8ab8-0737a2640b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.376009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 773/1553] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.377136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75766858-38e3-48f3-b4f2-9880e62aa041 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.377416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75766858-38e3-48f3-b4f2-9880e62aa041 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.377685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75766858-38e3-48f3-b4f2-9880e62aa041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.380987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17eaa6-2fc8-4679-8ab8-0737a2640b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.381669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17eaa6-2fc8-4679-8ab8-0737a2640b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.382700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-5e17eaa6-2fc8-4679-8ab8-0737a2640b92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:58:55.382700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:58:55.382700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:58:55.384047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17eaa6-2fc8-4679-8ab8-0737a2640b92 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.397619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-75766858-38e3-48f3-b4f2-9880e62aa041 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.398825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-75766858-38e3-48f3-b4f2-9880e62aa041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.407027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 774/1554] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.407603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75766858-38e3-48f3-b4f2-9880e62aa041 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.417688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75766858-38e3-48f3-b4f2-9880e62aa041 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.422490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c405f-9081-40a2-b615-5f4f1279ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.423126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c405f-9081-40a2-b615-5f4f1279ae38 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.423397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c405f-9081-40a2-b615-5f4f1279ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.424223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c405f-9081-40a2-b615-5f4f1279ae38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.424885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c405f-9081-40a2-b615-5f4f1279ae38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.426800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c405f-9081-40a2-b615-5f4f1279ae38 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.431881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 781/1555] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.431881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ab92a-0c75-42d1-bb35-53a66520a29b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.431881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ab92a-0c75-42d1-bb35-53a66520a29b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.431881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ab92a-0c75-42d1-bb35-53a66520a29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.452660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e72ab92a-0c75-42d1-bb35-53a66520a29b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.453901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e72ab92a-0c75-42d1-bb35-53a66520a29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.454587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 775/1556] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.455740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c66bc5-5103-45ba-9df3-a89a10b9ede1 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.456020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c66bc5-5103-45ba-9df3-a89a10b9ede1 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.456307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c66bc5-5103-45ba-9df3-a89a10b9ede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.477524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-73c66bc5-5103-45ba-9df3-a89a10b9ede1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.478707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-73c66bc5-5103-45ba-9df3-a89a10b9ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.514998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 782/1557] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.515833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ab92a-0c75-42d1-bb35-53a66520a29b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.527855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ab92a-0c75-42d1-bb35-53a66520a29b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.532618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1415cb6-f576-47a3-835b-195eb0ed81b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.533955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1415cb6-f576-47a3-835b-195eb0ed81b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.534243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1415cb6-f576-47a3-835b-195eb0ed81b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.534516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1415cb6-f576-47a3-835b-195eb0ed81b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.535162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1415cb6-f576-47a3-835b-195eb0ed81b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.535462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1415cb6-f576-47a3-835b-195eb0ed81b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.537183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1415cb6-f576-47a3-835b-195eb0ed81b2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.539826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 776/1558] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.540635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c66bc5-5103-45ba-9df3-a89a10b9ede1 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.552113 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 783/1559] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.552317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c66bc5-5103-45ba-9df3-a89a10b9ede1 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.552414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4b943-d25f-4e5f-923c-118483e3dc4a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.552414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4b943-d25f-4e5f-923c-118483e3dc4a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.552590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4b943-d25f-4e5f-923c-118483e3dc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.562581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.563197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.563473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.563744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.564658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.566864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.574926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-38c4b943-d25f-4e5f-923c-118483e3dc4a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.576143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-38c4b943-d25f-4e5f-923c-118483e3dc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.584381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 784/1560] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.585665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064e5c-226a-44e6-8d11-b4401497e6d8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.585975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064e5c-226a-44e6-8d11-b4401497e6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.586283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064e5c-226a-44e6-8d11-b4401497e6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.599816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 777/1561] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.599816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.607498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-70064e5c-226a-44e6-8d11-b4401497e6d8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:55.608681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-70064e5c-226a-44e6-8d11-b4401497e6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:55.649092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:55.661493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6642dae-8f8a-4432-9aa8-766d6d87ef19 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.665637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:55.667285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.667945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.668218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.668511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.669193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.670227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING py.warnings [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:58:55.670227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: warnings.warn(msg) Oct 03 15:58:55.670227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: Oct 03 15:58:55.671691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.676993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 785/1562] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:55.677820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70064e5c-226a-44e6-8d11-b4401497e6d8 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.687507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70064e5c-226a-44e6-8d11-b4401497e6d8 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.692946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.693785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.694164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.694529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.695362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.695800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 778/1563] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.696879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.698617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.706944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aa9a-8438-4993-a9f2-135e7ef1f61c admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.712638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.713321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.713667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.714069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.714690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.715050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.716620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.722490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 786/1564] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.723376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.732600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 779/1565] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.733430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.733565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac599c2c-fb65-4f88-9796-10e095e836bc admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.738288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.739046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.739323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.739587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.740521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.740846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.742439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.749919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbde69b-7e75-4960-befd-92b02089033e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.755561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 787/1566] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.756412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.765156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.765711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.765984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.766605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.767170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.770013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.773316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6267db-b27e-4e9d-bc3c-6667ebc60409 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.778155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.778836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.779190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.779523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.780247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.780657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.782357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.796051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 788/1567] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.797562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.799260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 780/1568] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.800019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.812416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30b8ef-11d2-4850-ae4c-e7892b8c6c8a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.865695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fd2b2-a8a6-4567-af25-bea9221abfe7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.868691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.869320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.869592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.869860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.870485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.870605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.871183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.871460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.871726 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.872393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:58:55.872507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.872689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.874319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.886678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 781/1569] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:58:55.887668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.894810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 789/1570] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.895679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.898736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae511f4-c805-46eb-bae1-8113dba717f6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.903332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.903923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.904194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.904460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.905127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.905529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9387b0-98d4-441d-99bb-c9b85de1ea8e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.906711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.910779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.911361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.911639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.911912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.913051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.914656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.917586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 782/1571] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:55.918540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.928670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80b328-f712-4716-a1ef-002d9b01c057 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.929183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 790/1572] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:55.929183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.933868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.934454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.934732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.934994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.935615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.937524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.939836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853c1cd-147a-4555-ad09-9d2ed6c0bb6e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:55.944660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:55.945274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:55.945556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:55.945827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:55.946453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:55.948366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:55.966966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 783/1573] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.968233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:55.984041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 791/1574] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:55.985137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.037223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51219817-5a39-4f56-b5fa-d9daf9061bb3 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.045874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:56.054349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeab11c-5c49-4bd6-8a83-e72deb4cc74d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.058934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.059586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.059790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.060072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.060677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.062631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.078613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:56.092561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 792/1575] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:56.093731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.097953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.098570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.098846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.099114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.099753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.101734 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.123399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 784/1576] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:55 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:56.124059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.134882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637fad1-de75-4051-afe4-8458060f7a02 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.161726 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cc33c-30aa-441b-b92a-0d2a6e82daee admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.166453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.167037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.167311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.167580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.168207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.170155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.193121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.193726 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.194000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.194296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.194938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.196536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.198391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 793/1577] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:56.199282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.208841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 785/1578] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.209951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.219774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63df1b3-dbe6-46a5-9cbb-9e489ccd718d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.263239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053341f-2468-4fa4-96e6-6c93458b3945 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.268204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.268837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.269115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.269387 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.270024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.271628 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.277642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.278258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.278541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.278970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.279612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.281240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.281767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 794/1579] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.282551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.292701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae384ff5-91a4-4a0a-8626-89afde2dfb00 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.297270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.297861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.298134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.302927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.302927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.303169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 786/1580] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.303169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.303307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.316857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df8707-0a6c-4d01-b4ea-e7509c8a4433 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.318580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.319379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.319662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.319925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.320583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.322570 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.325437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 795/1581] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:56.326246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.336947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4047d96-d665-4021-9bf3-d1f43dbb2f30 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.343658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.344250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.344512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.344994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.345604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.347689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.351140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 787/1582] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:56.351140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.362661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1a4d5-fa5c-400c-b668-008c18f7d5d9 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.367764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.368483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.368805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.369069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.369737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.370035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 796/1583] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:56.370847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.371341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.380870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9980f97-1272-4a26-bc5f-39e7b64fc33a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.386740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 788/1584] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.386740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.386961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.386961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.386961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.386961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.387371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.389358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.395195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248039-fa71-4f51-b9fb-a98546c9ca04 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.399817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.400400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.400677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.400959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.401594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.403480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.404999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:56.417123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 797/1585] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:56.417123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.417401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 789/1586] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.418766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.426146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec199bcc-1b93-4b7a-965f-823505bf11b1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.430782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62765ecc-eda6-4d7e-9450-4bed65b55535 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.431396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.432241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.432521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.432894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] PATH_INFO: `/v3/projects/a8db1f811cf84507ad7ea7058d6a0d25/users/73a1aa0996db4c8ea851b2ba45026b35/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.435420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.436061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.436339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.436612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] PATH_INFO: `/v3/projects/943f7e5ccad5410896a40678aeaaf1c1/users/105c6710d8994b0c9a1b66c4dad6dbc8/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.448803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8db1f811cf84507ad7ea7058d6a0d25, user_id=73a1aa0996db4c8ea851b2ba45026b35, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.449908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:58:56.449908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:58:56.449908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:58:56.452056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.452211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=943f7e5ccad5410896a40678aeaaf1c1, user_id=105c6710d8994b0c9a1b66c4dad6dbc8, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.454970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.461411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:56.467017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 798/1587] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:56 2022] PUT /identity/v3/projects/a8db1f811cf84507ad7ea7058d6a0d25/users/73a1aa0996db4c8ea851b2ba45026b35/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:56.468601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.469584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 790/1588] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:56 2022] PUT /identity/v3/projects/943f7e5ccad5410896a40678aeaaf1c1/users/105c6710d8994b0c9a1b66c4dad6dbc8/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:56.470745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.540345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c89b77-c4be-4379-a3dc-279a9da850b0 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.544151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50537918-c707-401e-adcf-96058d3a020e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.545116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.545724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.546004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.546275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.546920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.549124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.550473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.551233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.551571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.551858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.552532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.554208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.559014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 791/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.559834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.570604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 799/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.570604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.570888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d1559-e284-42b9-843d-17f0a74f4850 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.575970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.576574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.576891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.577167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.577816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.578539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80f1ac-20dc-4fd3-9926-0c29748f48c9 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.579419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.583281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.583930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.584209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.584477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.585139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.586715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.589569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 792/1591] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.590219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.597690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 800/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.598603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.600143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cdd15-1db0-462f-8106-61f55cd96a02 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.605112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951f9dc-381e-4069-ae97-9fe87fc3aa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.605707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f9dc-381e-4069-ae97-9fe87fc3aa4f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.605985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f9dc-381e-4069-ae97-9fe87fc3aa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.606258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f9dc-381e-4069-ae97-9fe87fc3aa4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.606897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951f9dc-381e-4069-ae97-9fe87fc3aa4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.608086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88795d-b019-43d0-be77-c8ebc541c3e7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.608484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951f9dc-381e-4069-ae97-9fe87fc3aa4f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.613384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb400caa-ea96-467f-a099-5ab9c81af69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.614026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb400caa-ea96-467f-a099-5ab9c81af69e admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.614282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb400caa-ea96-467f-a099-5ab9c81af69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.614556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb400caa-ea96-467f-a099-5ab9c81af69e admin admin] PATH_INFO: `/v3/projects/087dd438fedd40a78b6842b098111d48/users/3f564f91c5414974a12212414e51e182/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.619876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 793/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.621203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10bc11-e121-420a-b089-68b8bc98b49b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.621581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10bc11-e121-420a-b089-68b8bc98b49b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.621950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10bc11-e121-420a-b089-68b8bc98b49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.630122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb400caa-ea96-467f-a099-5ab9c81af69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=087dd438fedd40a78b6842b098111d48, user_id=3f564f91c5414974a12212414e51e182, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.632799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb400caa-ea96-467f-a099-5ab9c81af69e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.641900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ce10bc11-e121-420a-b089-68b8bc98b49b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:56.643303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ce10bc11-e121-420a-b089-68b8bc98b49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:56.646493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 801/1594] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:56 2022] PUT /identity/v3/projects/087dd438fedd40a78b6842b098111d48/users/3f564f91c5414974a12212414e51e182/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:56.647873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe1444-e0ca-4180-a6f8-b5ddcb6bbed9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.648562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe1444-e0ca-4180-a6f8-b5ddcb6bbed9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.648562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe1444-e0ca-4180-a6f8-b5ddcb6bbed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.668618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-94fe1444-e0ca-4180-a6f8-b5ddcb6bbed9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:56.669864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-94fe1444-e0ca-4180-a6f8-b5ddcb6bbed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:56.703996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 794/1595] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:56.705221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10bc11-e121-420a-b089-68b8bc98b49b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.715057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10bc11-e121-420a-b089-68b8bc98b49b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.719879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.720589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.721056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.721542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.722191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.723801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.737551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 795/1596] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.737551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.737830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 802/1597] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:56.737830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe1444-e0ca-4180-a6f8-b5ddcb6bbed9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.747390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb159a-5051-4854-a9fe-ac94afd339cf admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.750070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe1444-e0ca-4180-a6f8-b5ddcb6bbed9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.752882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.753664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.753936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.754208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.754842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.755666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.756266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.756431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.756534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.756839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.757479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.759093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.766619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 796/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.767463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.772381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 803/1599] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.772381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.777575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc069172-9039-4d19-9e8b-39685819c282 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.779588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a3c54-6add-4bfa-bdab-84fa4f012aa3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.782501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.783620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.783889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.784158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] PATH_INFO: `/v3/projects/30034fca82444cca9b98bc6385adba43/users/fa9fce4427904e59847ac2081c999bcb/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.784998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047d9a1-f2f3-4201-b5f3-689a3de9e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.785574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d9a1-f2f3-4201-b5f3-689a3de9e85e admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.785852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d9a1-f2f3-4201-b5f3-689a3de9e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.786125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d9a1-f2f3-4201-b5f3-689a3de9e85e admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.786751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047d9a1-f2f3-4201-b5f3-689a3de9e85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.788338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047d9a1-f2f3-4201-b5f3-689a3de9e85e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.798477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 804/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:56.799840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30034fca82444cca9b98bc6385adba43, user_id=fa9fce4427904e59847ac2081c999bcb, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.800842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248471-e67b-4673-bee0-a387b99adaa2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.801128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248471-e67b-4673-bee0-a387b99adaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.801396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248471-e67b-4673-bee0-a387b99adaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.802622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.812617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:56.818376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 797/1601] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:56 2022] PUT /identity/v3/projects/30034fca82444cca9b98bc6385adba43/users/fa9fce4427904e59847ac2081c999bcb/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:56.819442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.832332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cf248471-e67b-4673-bee0-a387b99adaa2 None None] MFA Rules not processed for user `3f564f91c5414974a12212414e51e182`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:56.833599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cf248471-e67b-4673-bee0-a387b99adaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:56.864113 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:56.877457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d761b-18b7-4e17-a761-ce318039f3c4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.881092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.881689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.881964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.882228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.900937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 805/1602] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:56.901749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf248471-e67b-4673-bee0-a387b99adaa2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.945132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.946562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.956351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 798/1603] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:56.957252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:56.966389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf248471-e67b-4673-bee0-a387b99adaa2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.966823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bdeaa-9166-4abc-ba99-b37dbcf16984 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:56.971908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.972532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.972761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:56.972893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.973097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.973543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:56.973826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:56.974097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] PATH_INFO: `/v3/projects/39b6cb58e2d24700b646769f8286c8f8/users/75cbfb46df4040c5babc47baff193f3e/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:56.993866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b6cb58e2d24700b646769f8286c8f8, user_id=75cbfb46df4040c5babc47baff193f3e, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.994238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:56.995944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:56.996752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.008849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 799/1604] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:57.009676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.010642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 806/1605] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:56 2022] PUT /identity/v3/projects/39b6cb58e2d24700b646769f8286c8f8/users/75cbfb46df4040c5babc47baff193f3e/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:57.011563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.026432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d5427-b5b2-45de-b8d8-440bb43f89bc service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.088898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587db28-ef91-4b14-bccd-fc132cdea224 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.094728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.095673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a810f92-6419-49b5-81a7-1dee6c95b312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.095821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.096252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.096559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] PATH_INFO: `/v3/projects/f2ed0fa1c2fe47a485c26522b2e81b8b/users/b3098a7103d94b29bc4bce30cfd05bc6/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.096974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a810f92-6419-49b5-81a7-1dee6c95b312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.097311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a810f92-6419-49b5-81a7-1dee6c95b312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.097590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a810f92-6419-49b5-81a7-1dee6c95b312 admin admin] PATH_INFO: `/v3/projects/d7ee0d51f58e4abd8867c54458f504fc/users/5e113c9ec968460ba070cfc1a69a25a1/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.114270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ed0fa1c2fe47a485c26522b2e81b8b, user_id=b3098a7103d94b29bc4bce30cfd05bc6, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.116068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a810f92-6419-49b5-81a7-1dee6c95b312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ee0d51f58e4abd8867c54458f504fc, user_id=5e113c9ec968460ba070cfc1a69a25a1, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.117197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.119686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a810f92-6419-49b5-81a7-1dee6c95b312 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.132947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 807/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:57 2022] PUT /identity/v3/projects/f2ed0fa1c2fe47a485c26522b2e81b8b/users/b3098a7103d94b29bc4bce30cfd05bc6/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:57.133781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.134594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 800/1607] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:57 2022] PUT /identity/v3/projects/d7ee0d51f58e4abd8867c54458f504fc/users/5e113c9ec968460ba070cfc1a69a25a1/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:57.135745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c08d9-21ce-40c2-b84a-e70cd1dfcedf None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.136079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c08d9-21ce-40c2-b84a-e70cd1dfcedf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.136606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c08d9-21ce-40c2-b84a-e70cd1dfcedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.178969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ab6c08d9-21ce-40c2-b84a-e70cd1dfcedf None None] MFA Rules not processed for user `fa9fce4427904e59847ac2081c999bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:57.180021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ab6c08d9-21ce-40c2-b84a-e70cd1dfcedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:57.185072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:57.190934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:57.214815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae1423-a5f3-447c-846a-29f7b18eb7b4 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.265172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 801/1608] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:57.265172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c08d9-21ce-40c2-b84a-e70cd1dfcedf None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.323165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633b2a8-efad-43f5-af4e-b675f305cd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.324950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633b2a8-efad-43f5-af4e-b675f305cd5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.325723 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633b2a8-efad-43f5-af4e-b675f305cd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.327829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633b2a8-efad-43f5-af4e-b675f305cd5d admin admin] PATH_INFO: `/v3/projects/1306948d947746d8a5aa02a2dac15ce0/users/cd7e6f61c4334b529c6d1b86b1f2b0bb/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.354256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633b2a8-efad-43f5-af4e-b675f305cd5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1306948d947746d8a5aa02a2dac15ce0, user_id=cd7e6f61c4334b529c6d1b86b1f2b0bb, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.357157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633b2a8-efad-43f5-af4e-b675f305cd5d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.368223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c08d9-21ce-40c2-b84a-e70cd1dfcedf None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.373125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 808/1609] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:57 2022] PUT /identity/v3/projects/1306948d947746d8a5aa02a2dac15ce0/users/cd7e6f61c4334b529c6d1b86b1f2b0bb/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:57.375424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a334-ce1a-4358-ba28-d2c943cb0e1e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.375720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a334-ce1a-4358-ba28-d2c943cb0e1e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.375993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1a334-ce1a-4358-ba28-d2c943cb0e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.402650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5ae1a334-ce1a-4358-ba28-d2c943cb0e1e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:57.404103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5ae1a334-ce1a-4358-ba28-d2c943cb0e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:57.438311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1555e30-75ba-4393-8504-b276ffb57fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.438920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555e30-75ba-4393-8504-b276ffb57fff admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.439372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555e30-75ba-4393-8504-b276ffb57fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.439690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1555e30-75ba-4393-8504-b276ffb57fff admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.440497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1555e30-75ba-4393-8504-b276ffb57fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.442224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1555e30-75ba-4393-8504-b276ffb57fff admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.475738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 802/1610] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:57.475738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f28c1-1b0b-4385-ba51-da0f569f16ac None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.475738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f28c1-1b0b-4385-ba51-da0f569f16ac None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.475738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f28c1-1b0b-4385-ba51-da0f569f16ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.479340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 809/1611] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:57.480740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40f380-7007-4303-a3e2-345c2a8d1616 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.481152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40f380-7007-4303-a3e2-345c2a8d1616 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.481530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40f380-7007-4303-a3e2-345c2a8d1616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.503374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2d40f380-7007-4303-a3e2-345c2a8d1616 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:57.504840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2d40f380-7007-4303-a3e2-345c2a8d1616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:57.505022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c07f28c1-1b0b-4385-ba51-da0f569f16ac None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:57.506182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c07f28c1-1b0b-4385-ba51-da0f569f16ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:57.562260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:57.591685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 810/1612] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:57.593280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848992b0-c41d-48f6-8052-6885ada20b19 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.594125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848992b0-c41d-48f6-8052-6885ada20b19 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.594609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848992b0-c41d-48f6-8052-6885ada20b19 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.596667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 803/1613] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:57.598112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06216bd-90b5-4839-833e-9471cceb6fa7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.598446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06216bd-90b5-4839-833e-9471cceb6fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.598720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06216bd-90b5-4839-833e-9471cceb6fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.602630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 811/1614] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 15:58:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:58:57.603683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848992b0-c41d-48f6-8052-6885ada20b19 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.619610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848992b0-c41d-48f6-8052-6885ada20b19 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.625650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.627211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.627906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.628703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.629742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.632307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.636331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e06216bd-90b5-4839-833e-9471cceb6fa7 None None] MFA Rules not processed for user `cd7e6f61c4334b529c6d1b86b1f2b0bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:57.637615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e06216bd-90b5-4839-833e-9471cceb6fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:57.641996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:57.671384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 812/1615] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/users => generated 297 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:57.674656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.768223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5e834-02f1-4321-9c68-71cc9958abd7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.772758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.773299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.773575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.773842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.789974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 804/1616] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:57.791273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12801e-367c-4e0a-a3f9-adf70acf1258 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.791667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12801e-367c-4e0a-a3f9-adf70acf1258 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.791942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12801e-367c-4e0a-a3f9-adf70acf1258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.793999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.795483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.808209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 813/1617] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:57.809398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.819335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b5371-812a-4c37-ba6a-ed09e85f9689 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.823186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.823782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.824052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.824317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.836042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.837533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.842419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-1a12801e-367c-4e0a-a3f9-adf70acf1258 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:57.844375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1a12801e-367c-4e0a-a3f9-adf70acf1258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:57.849879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 814/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:57.850786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.861761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f807b-0346-4dc2-a912-652db047bb5e service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.867183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.867848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.868125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.868510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.882180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.886975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.898668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 815/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:57.899675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.911989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb58f6-6d96-42cc-991e-7b6e3fa0b364 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.917981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.918738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.919162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.919647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.920324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.922283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.936216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 816/1620] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:57.937504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:57.952587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cc49d-556b-445f-8dc8-26507ac60dc1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:57.965368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:57.967604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:57.969132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:57.969648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:57.971898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:57.977393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:57.997333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 805/1621] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:58.000279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12801e-367c-4e0a-a3f9-adf70acf1258 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.027845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12801e-367c-4e0a-a3f9-adf70acf1258 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.044334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.045135 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.045341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.046804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] PATH_INFO: `/v3/projects/ca0f8ab41d9348c18cccb217a748011d/users/191ce0dbb04144098cc1f9ab11b74669/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.054831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 817/1622] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:58:57 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:58.056152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.086644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0f8ab41d9348c18cccb217a748011d, user_id=191ce0dbb04144098cc1f9ab11b74669, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.090069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.118027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 806/1623] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:57 2022] PUT /identity/v3/projects/ca0f8ab41d9348c18cccb217a748011d/users/191ce0dbb04144098cc1f9ab11b74669/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:58.119668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.186714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:58.225721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8ebbb-e395-47a4-a710-20e54c534ab1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.240792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcafd0-e52a-41b3-9875-f74e9fe48edb admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.247771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.247771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.248073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.248185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.248891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.251215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.284335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 807/1624] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:58:58 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:58:58.284335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.296621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:58.301316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53264ef5-233f-49d1-8998-1dc8f41b9240 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.307857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.308591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.308921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.310331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.311019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.317116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.324384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 808/1625] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:58.326343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.346082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca761e4-9632-4cbf-9864-be815faa4930 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.357991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.360492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.361889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.362311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.363406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.365979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.378963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 809/1626] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:58.379919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.382648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b42a2b-e92e-4c90-9585-8ac477187893 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.383609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b42a2b-e92e-4c90-9585-8ac477187893 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.384701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b42a2b-e92e-4c90-9585-8ac477187893 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.385238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b42a2b-e92e-4c90-9585-8ac477187893 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.392898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff6a2c-b9e7-424b-9aba-ac372bc32334 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.398758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.399426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.399745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.400022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] PATH_INFO: `/v3/projects/ca0f8ab41d9348c18cccb217a748011d/users/191ce0dbb04144098cc1f9ab11b74669/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.403007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0f8ab41d9348c18cccb217a748011d, user_id=191ce0dbb04144098cc1f9ab11b74669, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.406118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.430878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 810/1627] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:58 2022] PUT /identity/v3/projects/ca0f8ab41d9348c18cccb217a748011d/users/191ce0dbb04144098cc1f9ab11b74669/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:58.430878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.512836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e947c-3a7a-4c66-aa58-754d7ae4b3db admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.522388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.524128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.524528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.524964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] PATH_INFO: `/v3/projects/dd13bf032a064ffeb1a1d5e41e7519b5/users/10e2f645a24d40c78a15bc73d6f01c4c/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.541836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b42a2b-e92e-4c90-9585-8ac477187893 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.543639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b42a2b-e92e-4c90-9585-8ac477187893 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.546528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd13bf032a064ffeb1a1d5e41e7519b5, user_id=10e2f645a24d40c78a15bc73d6f01c4c, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.550808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.564297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 818/1628] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:58.566267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822366-89da-402e-b3e0-143dd64ec31f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.566552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822366-89da-402e-b3e0-143dd64ec31f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.566823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822366-89da-402e-b3e0-143dd64ec31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.569671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 811/1629] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:58 2022] PUT /identity/v3/projects/dd13bf032a064ffeb1a1d5e41e7519b5/users/10e2f645a24d40c78a15bc73d6f01c4c/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:58.575333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.625526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cc822366-89da-402e-b3e0-143dd64ec31f None None] MFA Rules not processed for user `191ce0dbb04144098cc1f9ab11b74669`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:58.626988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cc822366-89da-402e-b3e0-143dd64ec31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:58.771625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f28ec8-491f-453c-adaf-eaee3013f273 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.787494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.788532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.788876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.790213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.792245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.797010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 819/1630] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:58.797680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.809934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822366-89da-402e-b3e0-143dd64ec31f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.812476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:58.820923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 812/1631] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:58:58.825700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:58.840966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c266ed2-0236-446d-bbb1-fa9fa4dc0e5a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.849326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aac7f-c6d2-4682-a06b-a0034f797330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:58.850113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aac7f-c6d2-4682-a06b-a0034f797330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.850397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aac7f-c6d2-4682-a06b-a0034f797330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.850726 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aac7f-c6d2-4682-a06b-a0034f797330 admin admin] PATH_INFO: `/v3/projects/dd13bf032a064ffeb1a1d5e41e7519b5/users/10e2f645a24d40c78a15bc73d6f01c4c/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.853712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aac7f-c6d2-4682-a06b-a0034f797330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd13bf032a064ffeb1a1d5e41e7519b5, user_id=10e2f645a24d40c78a15bc73d6f01c4c, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:58.857991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38aac7f-c6d2-4682-a06b-a0034f797330 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:58.875195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 813/1632] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:58:58 2022] PUT /identity/v3/projects/dd13bf032a064ffeb1a1d5e41e7519b5/users/10e2f645a24d40c78a15bc73d6f01c4c/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:58:58.886426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6572-8f0a-48b9-88af-7ed2b5f17807 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:58.886873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6572-8f0a-48b9-88af-7ed2b5f17807 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:58.887344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf6572-8f0a-48b9-88af-7ed2b5f17807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:58.935971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822366-89da-402e-b3e0-143dd64ec31f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:58.939042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7dcf6572-8f0a-48b9-88af-7ed2b5f17807 None None] MFA Rules not processed for user `10e2f645a24d40c78a15bc73d6f01c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:58.941747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7dcf6572-8f0a-48b9-88af-7ed2b5f17807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:58.947053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:58:59.066164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:59.066988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:59.067270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:59.067542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:59.072205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 814/1633] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:59.073519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b9e55-134e-4350-99ec-06206124ef35 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:59.073797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b9e55-134e-4350-99ec-06206124ef35 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:59.074453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b9e55-134e-4350-99ec-06206124ef35 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:59.077426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 815/1634] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 15:58:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:58:59.087475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76694b16-ae27-413e-887f-e906445335d5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:59.087759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76694b16-ae27-413e-887f-e906445335d5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:59.088032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76694b16-ae27-413e-887f-e906445335d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:59.128949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-76694b16-ae27-413e-887f-e906445335d5 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:58:59.131701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-76694b16-ae27-413e-887f-e906445335d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:58:59.214423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:59.217810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:59.246529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 820/1635] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:58:59.247436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:59.280756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f2378-14a9-4dd3-99a8-adfde700390d service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:59.293680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:59.294395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:59.294674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:59.294948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:59.323205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:59.324754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:59.327103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 816/1636] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:58:59.349751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 821/1637] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:59.385536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76694b16-ae27-413e-887f-e906445335d5 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:59.398620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76694b16-ae27-413e-887f-e906445335d5 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:59.405067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:59.406547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:59.407048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:59.407573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:59.426390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:59.429002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:59.446493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 817/1638] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:58:59.578385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:58:59.590881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda458-5101-4f52-bcbe-edf80db60185 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:58:59.600565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2497a85-1c02-460b-9ed4-0c2da3d9b2be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:58:59.601293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2497a85-1c02-460b-9ed4-0c2da3d9b2be service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:58:59.601566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2497a85-1c02-460b-9ed4-0c2da3d9b2be service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:58:59.601832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2497a85-1c02-460b-9ed4-0c2da3d9b2be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:58:59.730110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2497a85-1c02-460b-9ed4-0c2da3d9b2be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:58:59.731585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2497a85-1c02-460b-9ed4-0c2da3d9b2be service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:58:59.753015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 822/1639] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:58:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:00.196475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:00.209756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f9953-7ff5-45b3-81d7-145d117fcbda service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:00.214778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f413c-fa8e-4763-bf03-ea2e84b36786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:00.215403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f413c-fa8e-4763-bf03-ea2e84b36786 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:00.215808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f413c-fa8e-4763-bf03-ea2e84b36786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:00.216215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f413c-fa8e-4763-bf03-ea2e84b36786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:00.324820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:00.362699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f413c-fa8e-4763-bf03-ea2e84b36786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:00.364423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f413c-fa8e-4763-bf03-ea2e84b36786 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:00.382434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 818/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:00.909095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d18ed-e89c-415a-9160-bf17a590a540 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:00.909750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d18ed-e89c-415a-9160-bf17a590a540 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:00.909750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d18ed-e89c-415a-9160-bf17a590a540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:00.984670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d10d18ed-e89c-415a-9160-bf17a590a540 None None] MFA Rules not processed for user `75cbfb46df4040c5babc47baff193f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:00.988404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d10d18ed-e89c-415a-9160-bf17a590a540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:01.118051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:01.198510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb99e-7f91-498b-94b5-97e81379341b None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.199076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb99e-7f91-498b-94b5-97e81379341b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.199076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb99e-7f91-498b-94b5-97e81379341b None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.201683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 819/1641] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 15:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:59:01.210262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc93f-071c-430c-b265-5bb4f81d94e3 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.210720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc93f-071c-430c-b265-5bb4f81d94e3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.211065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc93f-071c-430c-b265-5bb4f81d94e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.229373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 823/1642] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:01.230679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d18ed-e89c-415a-9160-bf17a590a540 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:01.249495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0fddc93f-071c-430c-b265-5bb4f81d94e3 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:01.252894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0fddc93f-071c-430c-b265-5bb4f81d94e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:01.281893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 820/1643] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 15:59:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:01.284238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc0547-271a-4951-8829-bba3ed774445 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.284871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc0547-271a-4951-8829-bba3ed774445 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.285154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc0547-271a-4951-8829-bba3ed774445 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.295672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 821/1644] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 15:59:01 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:59:01.300428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897100-03b0-462d-839d-77b3af68e12f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.300921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897100-03b0-462d-839d-77b3af68e12f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.301404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24897100-03b0-462d-839d-77b3af68e12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.369447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-24897100-03b0-462d-839d-77b3af68e12f None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:01.371517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-24897100-03b0-462d-839d-77b3af68e12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:01.391729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 822/1645] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 15:59:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:01.411347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897100-03b0-462d-839d-77b3af68e12f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:01.414449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d18ed-e89c-415a-9160-bf17a590a540 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:01.420075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:01.421114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.421663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.421961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.424107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:01.428636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:01.429389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24897100-03b0-462d-839d-77b3af68e12f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:01.457883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:01.458612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.459465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.460982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.474652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:01.477416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:01.482961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 824/1646] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:01 2022] POST /identity/v3/projects => generated 405 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:01.487661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:01.523539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 823/1647] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:01.593114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36f29-7751-4e0b-8ca0-67428f420a76 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:01.603516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:01.604386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.604853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.605138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.606126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:01.608644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:01.637661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 825/1648] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:01 2022] POST /identity/v3/users => generated 309 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:01.641088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:01.687117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec9c79-5239-42de-80c5-ddef7d4f2e87 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:01.687117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:01.687117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.687117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.693490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.693490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:01.693490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:01.713256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 824/1649] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:01.717022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:01.735087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c022a73-67a7-416f-a018-80810202c7d1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:01.755075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:01.755798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.756082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.756360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] PATH_INFO: `/v3/projects/a58dda0938e84bb892dc5c697fb0fe8c/users/c774a2fe964a4448bb6794d6acfdd25b/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.788021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58dda0938e84bb892dc5c697fb0fe8c, user_id=c774a2fe964a4448bb6794d6acfdd25b, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:01.791430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:01.820854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 826/1650] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:01 2022] PUT /identity/v3/projects/a58dda0938e84bb892dc5c697fb0fe8c/users/c774a2fe964a4448bb6794d6acfdd25b/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:01.822080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:01.833818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ad90a-96e8-48cc-8af4-0396f22a5b5d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:01.982970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b9884-ce3c-49d5-821f-ff352f1315e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:01.983721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9884-ce3c-49d5-821f-ff352f1315e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:01.984045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9884-ce3c-49d5-821f-ff352f1315e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:01.984327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b9884-ce3c-49d5-821f-ff352f1315e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:01.987175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b9884-ce3c-49d5-821f-ff352f1315e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:01.989751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b9884-ce3c-49d5-821f-ff352f1315e3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:02.018204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 825/1651] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:02.021689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:02.036223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43365366-f0bf-4581-92aa-f1b8b761d3a1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:02.049328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:02.050111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:02.050400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:02.050706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] PATH_INFO: `/v3/projects/a58dda0938e84bb892dc5c697fb0fe8c/users/c774a2fe964a4448bb6794d6acfdd25b/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:02.056681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58dda0938e84bb892dc5c697fb0fe8c, user_id=c774a2fe964a4448bb6794d6acfdd25b, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:02.067775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:02.104769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 827/1652] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:02 2022] PUT /identity/v3/projects/a58dda0938e84bb892dc5c697fb0fe8c/users/c774a2fe964a4448bb6794d6acfdd25b/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:05.025412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11911a-9a93-4394-868c-171b1b33fd01 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:05.027274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11911a-9a93-4394-868c-171b1b33fd01 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:05.028067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11911a-9a93-4394-868c-171b1b33fd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:05.130102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-eb11911a-9a93-4394-868c-171b1b33fd01 None None] MFA Rules not processed for user `c774a2fe964a4448bb6794d6acfdd25b`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:05.137310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-eb11911a-9a93-4394-868c-171b1b33fd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:05.288163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 826/1653] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:05.961791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:06.006747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:06.185718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487264f1-ddf0-4494-95be-f68d5758bd31 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:06.206989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:06.207669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:06.207950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:06.208276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] PATH_INFO: `/v3/users/3f564f91c5414974a12212414e51e182` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:06.210315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f564f91c5414974a12212414e51e182)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:06.215750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: WARNING py.warnings [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:59:06.215750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: warnings.warn(msg) Oct 03 15:59:06.215750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: Oct 03 15:59:06.225479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:06.424169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f564f91c5414974a12212414e51e182'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:06.426232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f564f91c5414974a12212414e51e182'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:06.441958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f564f91c5414974a12212414e51e182'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:06.492380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f564f91c5414974a12212414e51e182'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:06.522340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f564f91c5414974a12212414e51e182'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:06.541760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 828/1654] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:05 2022] DELETE /identity/v3/users/3f564f91c5414974a12212414e51e182 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:06.546269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee458d6-3d63-479a-8b92-b126e44885d9 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:06.546762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee458d6-3d63-479a-8b92-b126e44885d9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:06.546832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee458d6-3d63-479a-8b92-b126e44885d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:06.586779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6ee458d6-3d63-479a-8b92-b126e44885d9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:06.588046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6ee458d6-3d63-479a-8b92-b126e44885d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:06.686085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:06.769435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 827/1655] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:06.887274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:06.889090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:06.981242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924737a6-888e-4c8c-b63d-6faf7f6fd34f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:06.988582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649f672-0766-4d4d-8b9e-aa03c33c4c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:06.989784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649f672-0766-4d4d-8b9e-aa03c33c4c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:06.990163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649f672-0766-4d4d-8b9e-aa03c33c4c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:06.990477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649f672-0766-4d4d-8b9e-aa03c33c4c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:07.003911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649f672-0766-4d4d-8b9e-aa03c33c4c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:07.004138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649f672-0766-4d4d-8b9e-aa03c33c4c78 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:07.019737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 829/1656] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:07.402751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4fd9f-c360-4889-84e3-7ecc1ab1c78f None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:07.403836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4fd9f-c360-4889-84e3-7ecc1ab1c78f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:07.404348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4fd9f-c360-4889-84e3-7ecc1ab1c78f None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:07.406720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 828/1657] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 15:59:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:59:07.481597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:07.482042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:07.482562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:07.538153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:07.539434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:07.683386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:07.795206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 830/1658] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 15:59:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:07.981779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4fd9f-c360-4889-84e3-7ecc1ab1c78f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:07.985228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ecf4fd9f-c360-4889-84e3-7ecc1ab1c78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:08.022332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4fd9f-c360-4889-84e3-7ecc1ab1c78f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:08.028781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:08.029600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:08.030014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:08.030444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:08.050809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:08.052459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:08.089234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 829/1659] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:09.223880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:09.225913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:09.260509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64944ff3-a1a6-48a7-89ab-0c7a8afc19a7 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:09.278841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:09.279857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:09.280357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:09.280839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] PATH_INFO: `/v3/projects/087dd438fedd40a78b6842b098111d48` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:09.284123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087dd438fedd40a78b6842b098111d48)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:09.287126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:09.445048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087dd438fedd40a78b6842b098111d48', 'request_id': 'req-59f891c6-a504-4d2f-8cac-faf678b6bfd4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.474476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087dd438fedd40a78b6842b098111d48', 'request_id': 'req-59f891c6-a504-4d2f-8cac-faf678b6bfd4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.491085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087dd438fedd40a78b6842b098111d48', 'request_id': 'req-59f891c6-a504-4d2f-8cac-faf678b6bfd4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.514285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] The token cache is being invalidate because project 087dd438fedd40a78b6842b098111d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:09.514579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.515570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59f891c6-a504-4d2f-8cac-faf678b6bfd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.524888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 831/1660] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:09 2022] DELETE /identity/v3/projects/087dd438fedd40a78b6842b098111d48 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:09.528631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:09.547026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8179886-df0e-4ed0-a4c2-02c5f8d34a31 service placement] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:09.645509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:09.646168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:09.646446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:09.646715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] PATH_INFO: `/v3/users/191ce0dbb04144098cc1f9ab11b74669` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:09.649799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=191ce0dbb04144098cc1f9ab11b74669)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:09.654251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:09.763625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '191ce0dbb04144098cc1f9ab11b74669'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.780507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '191ce0dbb04144098cc1f9ab11b74669'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.805942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '191ce0dbb04144098cc1f9ab11b74669'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.827225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '191ce0dbb04144098cc1f9ab11b74669'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.827905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6face514-729b-4732-ae9e-48ea77cb38a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '191ce0dbb04144098cc1f9ab11b74669'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:09.843785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 830/1661] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:09 2022] DELETE /identity/v3/users/191ce0dbb04144098cc1f9ab11b74669 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:09.854663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:09.854943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:09.855216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:09.914181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] MFA Rules not processed for user `105c6710d8994b0c9a1b66c4dad6dbc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:09.915850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:10.055109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 832/1662] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:10.056149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:10.056689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:10.057081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:10.104286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] MFA Rules not processed for user `105c6710d8994b0c9a1b66c4dad6dbc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:10.107233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:10.129080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 831/1663] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:10.135757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:10.139937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:10.239800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:10.345600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d349f-8e58-476f-aed0-e9c7205d1b4c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:10.363703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:10.364399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:10.364679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:10.365768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:10.386753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:10.388519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:10.428608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 833/1664] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:10.540932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:10.549015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:10.678767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:10.815308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e338015-0c8f-45fb-b16f-c5c3f3423f17 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:10.827896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:10.828797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:10.829860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:10.830234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] PATH_INFO: `/v3/projects/ca0f8ab41d9348c18cccb217a748011d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:10.831315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0f8ab41d9348c18cccb217a748011d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:10.832345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: WARNING py.warnings [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 15:59:10.832345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: warnings.warn(msg) Oct 03 15:59:10.832345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: Oct 03 15:59:10.833793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:10.900919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:10.901955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:10.929379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0adc0d3-9e0d-4950-af7c-bb9baf5d7912 service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:11.000512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0f8ab41d9348c18cccb217a748011d', 'request_id': 'req-4d03e610-d9f9-4534-bbb6-b360e9f3de35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.028281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0f8ab41d9348c18cccb217a748011d', 'request_id': 'req-4d03e610-d9f9-4534-bbb6-b360e9f3de35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.049580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0f8ab41d9348c18cccb217a748011d', 'request_id': 'req-4d03e610-d9f9-4534-bbb6-b360e9f3de35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.095911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] The token cache is being invalidate because project ca0f8ab41d9348c18cccb217a748011d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:11.096433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.097229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4d03e610-d9f9-4534-bbb6-b360e9f3de35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.099298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 832/1665] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:10 2022] DELETE /identity/v3/projects/ca0f8ab41d9348c18cccb217a748011d => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:11.105411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:11.108321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:11.110623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:11.142690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:11.154268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:11.157962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:11.158404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] PATH_INFO: `/v3/users/fa9fce4427904e59847ac2081c999bcb` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:11.161078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9fce4427904e59847ac2081c999bcb)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:11.164331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:11.179498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:11.183063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:11.223203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 833/1666] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:11.224373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:11.227073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:11.281367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391686ca-9c6f-43e8-80e0-230f719a7252 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:11.307733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:11.309042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:11.309929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:11.311804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] PATH_INFO: `/v3/users/75cbfb46df4040c5babc47baff193f3e` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:11.316049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75cbfb46df4040c5babc47baff193f3e)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:11.318332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:11.369956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9fce4427904e59847ac2081c999bcb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.371746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9fce4427904e59847ac2081c999bcb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.384865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9fce4427904e59847ac2081c999bcb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.425355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9fce4427904e59847ac2081c999bcb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.433195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75cbfb46df4040c5babc47baff193f3e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.441911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e42a8ed3-bee9-45a1-a29a-103e98ba87b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9fce4427904e59847ac2081c999bcb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.448527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75cbfb46df4040c5babc47baff193f3e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.463011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 834/1667] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:10 2022] DELETE /identity/v3/users/fa9fce4427904e59847ac2081c999bcb => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:11.465837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53752-52f7-46a3-bbdc-ce408a6696d9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:11.466493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53752-52f7-46a3-bbdc-ce408a6696d9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:11.469107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d53752-52f7-46a3-bbdc-ce408a6696d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:11.480408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75cbfb46df4040c5babc47baff193f3e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.492374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75cbfb46df4040c5babc47baff193f3e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.493189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d564ce49-6133-4b33-b130-12e92e6d7d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75cbfb46df4040c5babc47baff193f3e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:11.509168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 834/1668] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:11 2022] DELETE /identity/v3/users/75cbfb46df4040c5babc47baff193f3e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:11.509168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2166cb-bfab-4156-9b19-32f765796e4c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:11.509168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2166cb-bfab-4156-9b19-32f765796e4c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:11.509168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2166cb-bfab-4156-9b19-32f765796e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:11.522802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e1d53752-52f7-46a3-bbdc-ce408a6696d9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:11.524432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e1d53752-52f7-46a3-bbdc-ce408a6696d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:11.545784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bd2166cb-bfab-4156-9b19-32f765796e4c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:11.549900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bd2166cb-bfab-4156-9b19-32f765796e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:11.599442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:11.661581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 835/1669] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:11.671373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:11.715104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 835/1670] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:11.723061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2166cb-bfab-4156-9b19-32f765796e4c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:11.738263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2166cb-bfab-4156-9b19-32f765796e4c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:11.747043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:11.747741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:11.748026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:11.748556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:11.749456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:11.749900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:11.754452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:11.779452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53752-52f7-46a3-bbdc-ce408a6696d9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:11.790300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 836/1671] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:11.795132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:11.796074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d53752-52f7-46a3-bbdc-ce408a6696d9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:11.798877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:11.809863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:11.811994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:11.812282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:11.812557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:11.813262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:11.823251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:11.954779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 836/1672] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:11 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:11.962725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:12.045909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6127e7-26e3-40b7-9d7b-94c409c44017 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:12.178001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bd8c7-a6b5-47e6-b1dc-e78fe7613908 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:12.190254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:12.191058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.193248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.193499 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.194163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:12.200013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:12.260391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 837/1673] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:11 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:12.264289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:12.286683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca067d04-e317-410a-8d27-3063598cbe2f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:12.301112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:12.301922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.302340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:12.302527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.302933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.303039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.303252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.303527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.304134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:12.306564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:12.334518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 838/1674] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:59:12.340201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:12.349827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:12.351651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:12.375647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ab32e-dbfd-43aa-b6ab-7431d9acb551 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:12.379473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 837/1675] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:12.390469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd53048-3f32-4b7d-9f90-e1858a5c859e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:12.391333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53048-3f32-4b7d-9f90-e1858a5c859e admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.392327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53048-3f32-4b7d-9f90-e1858a5c859e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.392777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd53048-3f32-4b7d-9f90-e1858a5c859e admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.394838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53048-3f32-4b7d-9f90-e1858a5c859e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:12.396992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd53048-3f32-4b7d-9f90-e1858a5c859e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:12.421586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 839/1676] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:12.426334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:12.451889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36805b1-f810-4649-b016-ed4de9f36b61 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:12.460587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5968f64-ce7a-498c-85ea-7f7275fd9585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:12.463051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968f64-ce7a-498c-85ea-7f7275fd9585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.463342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968f64-ce7a-498c-85ea-7f7275fd9585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.463620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968f64-ce7a-498c-85ea-7f7275fd9585 admin admin] PATH_INFO: `/v3/projects/1bf3a255785444ee8ed1f55d2f2d35f1/users/ec3fd07252b94b82958093c70d31e5e8/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.515085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df30e4-2406-4419-8223-87007bf46b91 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.515496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df30e4-2406-4419-8223-87007bf46b91 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.515875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df30e4-2406-4419-8223-87007bf46b91 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.518931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 840/1677] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 15:59:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:59:12.526850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc8c6-1399-41c4-8cb7-b15c247e8ac5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.527280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc8c6-1399-41c4-8cb7-b15c247e8ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.528029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc8c6-1399-41c4-8cb7-b15c247e8ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.532663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5968f64-ce7a-498c-85ea-7f7275fd9585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf3a255785444ee8ed1f55d2f2d35f1, user_id=ec3fd07252b94b82958093c70d31e5e8, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:12.540856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5968f64-ce7a-498c-85ea-7f7275fd9585 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:12.587784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 838/1678] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:12 2022] PUT /identity/v3/projects/1bf3a255785444ee8ed1f55d2f2d35f1/users/ec3fd07252b94b82958093c70d31e5e8/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:12.590321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.592860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.593392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.625819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-230dc8c6-1399-41c4-8cb7-b15c247e8ac5 None None] MFA Rules not processed for user `2d007a3d205345aebcc36a36d039da2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:12.628634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-230dc8c6-1399-41c4-8cb7-b15c247e8ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:12.677792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] MFA Rules not processed for user `ec3fd07252b94b82958093c70d31e5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:12.683414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:12.772624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 841/1679] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 15:59:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:12.806586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:12.856424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dc8c6-1399-41c4-8cb7-b15c247e8ac5 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:12.874884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dc8c6-1399-41c4-8cb7-b15c247e8ac5 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:12.883470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:12.884107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:12.884385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:12.884658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:12.885020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 839/1680] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:12.896973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:12.898329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:12.913605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:12.915546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:12.939291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 842/1681] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:13.080521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022aec4-47bf-49c2-98a2-41cabe6219f3 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:13.090679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:13.093117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:13.095693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:13.096131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:13.116228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:13.119867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:13.170057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:13.171740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:13.180357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 840/1682] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:13.212266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:13.233268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:13.317442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed5360-81b5-4fae-af41-dc3c4c7ad990 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:13.323327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:13.325305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:13.328391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:13.330125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] PATH_INFO: `/v3/users/ec3fd07252b94b82958093c70d31e5e8` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:13.332152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec3fd07252b94b82958093c70d31e5e8)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:13.335375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:13.434966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11055569-5ec4-40a7-9be3-436f9bcbe822 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:13.461530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:13.462205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:13.462483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:13.462754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] PATH_INFO: `/v3/users/cd7e6f61c4334b529c6d1b86b1f2b0bb` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:13.465677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7e6f61c4334b529c6d1b86b1f2b0bb)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:13.467953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:13.485780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec3fd07252b94b82958093c70d31e5e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.501806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec3fd07252b94b82958093c70d31e5e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.533263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec3fd07252b94b82958093c70d31e5e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.556650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec3fd07252b94b82958093c70d31e5e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.557762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec3fd07252b94b82958093c70d31e5e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.578378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 841/1683] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:13 2022] DELETE /identity/v3/users/ec3fd07252b94b82958093c70d31e5e8 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:13.578378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:13.579773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:13.592099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:13.618761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7e6f61c4334b529c6d1b86b1f2b0bb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.619759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7e6f61c4334b529c6d1b86b1f2b0bb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.630528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7e6f61c4334b529c6d1b86b1f2b0bb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.655017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7e6f61c4334b529c6d1b86b1f2b0bb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.671907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-62fa2a0a-8dd6-4138-bf97-6116b67773a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7e6f61c4334b529c6d1b86b1f2b0bb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:13.683731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 843/1684] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:13 2022] DELETE /identity/v3/users/cd7e6f61c4334b529c6d1b86b1f2b0bb => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:13.685133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f4e3f-509c-4162-80e7-b8843b4758bc None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:13.685505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f4e3f-509c-4162-80e7-b8843b4758bc None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:13.685657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f4e3f-509c-4162-80e7-b8843b4758bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:13.734167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-868f4e3f-509c-4162-80e7-b8843b4758bc None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:13.738637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-868f4e3f-509c-4162-80e7-b8843b4758bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:13.752229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422012b1-45f7-48c6-99b1-6ff54839c66d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:13.867387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:13.869439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:13.869439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:13.869958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] PATH_INFO: `/v3/projects/39b6cb58e2d24700b646769f8286c8f8` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:13.873925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 844/1685] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:13.876114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:13.876578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:13.876972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:13.886397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b6cb58e2d24700b646769f8286c8f8)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:13.917568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:13.937680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:13.943565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:13.957911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 845/1686] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:13.958935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:13.973876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:13.991572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee7c9-dd4b-4df7-bc80-059ba163558d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:14.003603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:14.004454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:14.004888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:14.005804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] PATH_INFO: `/v3/projects/30034fca82444cca9b98bc6385adba43` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:14.006725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30034fca82444cca9b98bc6385adba43)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:14.010079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:14.051101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:14.104941 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b6cb58e2d24700b646769f8286c8f8', 'request_id': 'req-05889510-cd37-426e-b242-4d4b02bf113d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.107210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30034fca82444cca9b98bc6385adba43', 'request_id': 'req-73257dfc-4fb8-4e52-9420-21dab0b48e72'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.125269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b6cb58e2d24700b646769f8286c8f8', 'request_id': 'req-05889510-cd37-426e-b242-4d4b02bf113d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.134643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b6cb58e2d24700b646769f8286c8f8', 'request_id': 'req-05889510-cd37-426e-b242-4d4b02bf113d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.142366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30034fca82444cca9b98bc6385adba43', 'request_id': 'req-73257dfc-4fb8-4e52-9420-21dab0b48e72'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.165078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30034fca82444cca9b98bc6385adba43', 'request_id': 'req-73257dfc-4fb8-4e52-9420-21dab0b48e72'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.168248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] The token cache is being invalidate because project 39b6cb58e2d24700b646769f8286c8f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:14.168548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.178068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.185102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 842/1687] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:13 2022] DELETE /identity/v3/projects/39b6cb58e2d24700b646769f8286c8f8 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:14.186181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:14.193427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.203216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] The token cache is being invalidate because project 30034fca82444cca9b98bc6385adba43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:14.203644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.205655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.221759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 846/1688] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:13 2022] DELETE /identity/v3/projects/30034fca82444cca9b98bc6385adba43 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:14.222678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:14.237928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.342689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:14.351654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73257dfc-4fb8-4e52-9420-21dab0b48e72 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:14.357631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:14.359340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:14.359816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:14.360321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:14.365566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.385040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05889510-cd37-426e-b242-4d4b02bf113d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:14.395629 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313235c-21a2-496f-9c7b-38b333ae1766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:14.399595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313235c-21a2-496f-9c7b-38b333ae1766 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:14.399991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313235c-21a2-496f-9c7b-38b333ae1766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:14.400368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313235c-21a2-496f-9c7b-38b333ae1766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:14.406106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3313235c-21a2-496f-9c7b-38b333ae1766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.514363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:14.519017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:14.557814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 847/1689] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:14.558802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:14.567244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.597902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313235c-21a2-496f-9c7b-38b333ae1766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:14.605073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313235c-21a2-496f-9c7b-38b333ae1766 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:14.617355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dddff9-d428-4b18-a7d2-aa26eb93ff13 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:14.629344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:14.630334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:14.630831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:14.631341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] PATH_INFO: `/v3/users/c774a2fe964a4448bb6794d6acfdd25b` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:14.633063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] RBAC: Authorizing `identity:delete_user(user_id=c774a2fe964a4448bb6794d6acfdd25b)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:14.635567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:14.638610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 843/1690] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:14.640088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:14.640379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:14.643151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:14.695263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:14.699206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.721357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 844/1691] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:14.730299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:14.734473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.797767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c774a2fe964a4448bb6794d6acfdd25b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.800107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c774a2fe964a4448bb6794d6acfdd25b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.823110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c774a2fe964a4448bb6794d6acfdd25b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.828072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:14.863284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c774a2fe964a4448bb6794d6acfdd25b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.893304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-38e16917-9441-4221-9ddb-34c5865484af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c774a2fe964a4448bb6794d6acfdd25b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:14.905019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 848/1692] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:14 2022] DELETE /identity/v3/users/c774a2fe964a4448bb6794d6acfdd25b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:14.906343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453de4-1792-4312-b0f6-68f8c0984197 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:14.906630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453de4-1792-4312-b0f6-68f8c0984197 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:14.906907 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81453de4-1792-4312-b0f6-68f8c0984197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:14.955581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-81453de4-1792-4312-b0f6-68f8c0984197 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:14.956910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-81453de4-1792-4312-b0f6-68f8c0984197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:14.975911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd6c83-1a46-4426-8a2d-4cae48517a17 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:15.156185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 849/1693] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:15.158601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.158975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.159214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.162919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:15.164049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.164439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.164948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.170814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:15.202206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:15.204097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:15.229644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] MFA Rules not processed for user `73a1aa0996db4c8ea851b2ba45026b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:15.231021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:15.246851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 845/1694] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:15.249067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:15.279600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd33d7-d917-43bf-b979-d2e7a23c901b service placement] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:15.291778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:15.293159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.293159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.293159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.293605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:15.293910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:15.297703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:15.340951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 846/1695] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:15.349719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:15.355892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:15.365277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cb4b9-e9b7-4302-a73c-9a13b1401a4b admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:15.395718 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:15.403680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.404043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.404382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.405216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:15.408622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:15.440234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 850/1696] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:15.458793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:15.459922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:15.490511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 847/1697] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:15 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:15.495067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:15.624055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:15.640474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e011e-128c-4ddf-8c95-7c1a6aaf7fc6 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:15.706280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7bb5d-20b3-4ab9-9c06-2fba2b73c48d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:15.715472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:15.717794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.718091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.718627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.719284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:15.726108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:15.787042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 848/1698] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:15 2022] POST /identity/v3/users => generated 286 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:15.792615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:15.829420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4de7b-32b3-4b2e-a15b-78f63777a07b admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:15.830331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:15.831809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.832844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.834053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.852194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:15.856358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.856807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.857477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.859439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:15.861866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:15.914372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 849/1699] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:15.919519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:15.951431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d73bb2-3df4-447c-8be6-747d5d596551 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:15.964476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:15.965323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:15.967508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:15.967807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:15.973472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:15.983098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:16.011472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 850/1700] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:16.012175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:16.019617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:16.057310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778f099-9772-42e7-9e5a-25becbef5962 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:16.065696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:16.067228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:16.067664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:16.068078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] PATH_INFO: `/v3/projects/1bf3a255785444ee8ed1f55d2f2d35f1` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:16.069770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf3a255785444ee8ed1f55d2f2d35f1)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:16.079601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:16.082592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:16.083990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:16.129150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 851/1701] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:16.130358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:16.169340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f2de-bc51-4e85-ba4a-0aeaf7e9e3fe service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:16.193676 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de388ae-ab64-4f52-87b4-682698722c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:16.194763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de388ae-ab64-4f52-87b4-682698722c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:16.194763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de388ae-ab64-4f52-87b4-682698722c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:16.195004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de388ae-ab64-4f52-87b4-682698722c43 admin admin] PATH_INFO: `/v3/projects/ca133ddf3c61490093232365a8555999/users/aaafa02a240c4d389288dcfc7aefb471/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:16.205056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf3a255785444ee8ed1f55d2f2d35f1', 'request_id': 'req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.219261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf3a255785444ee8ed1f55d2f2d35f1', 'request_id': 'req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.240177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf3a255785444ee8ed1f55d2f2d35f1', 'request_id': 'req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.257509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] The token cache is being invalidate because project 1bf3a255785444ee8ed1f55d2f2d35f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:16.257938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.258710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.265873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 851/1702] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:16 2022] DELETE /identity/v3/projects/1bf3a255785444ee8ed1f55d2f2d35f1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:16.267589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:16.269236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:16.273622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de388ae-ab64-4f52-87b4-682698722c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca133ddf3c61490093232365a8555999, user_id=aaafa02a240c4d389288dcfc7aefb471, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:16.276372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de388ae-ab64-4f52-87b4-682698722c43 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:16.361166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 852/1703] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:16 2022] PUT /identity/v3/projects/ca133ddf3c61490093232365a8555999/users/aaafa02a240c4d389288dcfc7aefb471/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:16.371604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92abdd8-2e3b-4f2d-bcd7-52ab4d557c97 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:16.371968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92abdd8-2e3b-4f2d-bcd7-52ab4d557c97 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:16.372392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92abdd8-2e3b-4f2d-bcd7-52ab4d557c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:16.406626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:16.426177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-f92abdd8-2e3b-4f2d-bcd7-52ab4d557c97 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:16.427416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f92abdd8-2e3b-4f2d-bcd7-52ab4d557c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:16.512109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb1fb9-9a1a-4085-93b1-fafa187de2ec admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:16.655350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 853/1704] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:16.657644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b7c0b-101c-4adc-bdf6-421753f4f1d4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:16.657926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b7c0b-101c-4adc-bdf6-421753f4f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:16.658187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b7c0b-101c-4adc-bdf6-421753f4f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:16.721149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:16.721859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:16.722841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e29b7c0b-101c-4adc-bdf6-421753f4f1d4 None None] MFA Rules not processed for user `aaafa02a240c4d389288dcfc7aefb471`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:16.726267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:16.726267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] PATH_INFO: `/v3/projects/a58dda0938e84bb892dc5c697fb0fe8c` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:16.727426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e29b7c0b-101c-4adc-bdf6-421753f4f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:16.727908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58dda0938e84bb892dc5c697fb0fe8c)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:16.733806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:16.733977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:16.874486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 854/1705] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:16.877135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:16.892948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:16.893239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:16.893931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58dda0938e84bb892dc5c697fb0fe8c', 'request_id': 'req-1ddf1a8f-3b15-4e64-9c08-beebef299601'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.917284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58dda0938e84bb892dc5c697fb0fe8c', 'request_id': 'req-1ddf1a8f-3b15-4e64-9c08-beebef299601'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.927372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58dda0938e84bb892dc5c697fb0fe8c', 'request_id': 'req-1ddf1a8f-3b15-4e64-9c08-beebef299601'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.948289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] The token cache is being invalidate because project a58dda0938e84bb892dc5c697fb0fe8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:16.948761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.951788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:16.953540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 852/1706] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:16 2022] DELETE /identity/v3/projects/a58dda0938e84bb892dc5c697fb0fe8c => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:16.954820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:16.958699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:16.959153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:16.961156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:17.127727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 855/1707] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:17.128686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:17.134187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:17.207089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf1a8f-3b15-4e64-9c08-beebef299601 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:17.222208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:17.223375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:17.223849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:17.224308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] PATH_INFO: `/v3/projects/1306948d947746d8a5aa02a2dac15ce0` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:17.225777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1306948d947746d8a5aa02a2dac15ce0)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:17.228041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:17.286815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:17.396002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1306948d947746d8a5aa02a2dac15ce0', 'request_id': 'req-b2304122-3ad9-4d88-8609-6ef4be09b538'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:17.402100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0db4f-b375-4cb4-8bbc-8d3f703927f7 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:17.422782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1306948d947746d8a5aa02a2dac15ce0', 'request_id': 'req-b2304122-3ad9-4d88-8609-6ef4be09b538'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:17.435332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1306948d947746d8a5aa02a2dac15ce0', 'request_id': 'req-b2304122-3ad9-4d88-8609-6ef4be09b538'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:17.465145 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] The token cache is being invalidate because project 1306948d947746d8a5aa02a2dac15ce0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:17.465439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:17.467030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b2304122-3ad9-4d88-8609-6ef4be09b538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:17.476350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 853/1708] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:16 2022] DELETE /identity/v3/projects/1306948d947746d8a5aa02a2dac15ce0 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:17.476350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9867d99-9738-46e2-960e-666adf5e4b26 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:17.476350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9867d99-9738-46e2-960e-666adf5e4b26 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:17.477221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9867d99-9738-46e2-960e-666adf5e4b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:17.518262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:17.548580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e9867d99-9738-46e2-960e-666adf5e4b26 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:17.550012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e9867d99-9738-46e2-960e-666adf5e4b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:17.586497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:17.587956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:17.588242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:17.588519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:17.590706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:17.713014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 854/1709] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:17.716891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5018de-4784-44c2-9acc-22634bf32e47 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:17.717530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5018de-4784-44c2-9acc-22634bf32e47 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:17.719315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5018de-4784-44c2-9acc-22634bf32e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:17.740581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:17.751288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:17.771148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 856/1710] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:17.772485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:17.777239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:17.789284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-df5018de-4784-44c2-9acc-22634bf32e47 None None] MFA Rules not processed for user `b3098a7103d94b29bc4bce30cfd05bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:17.791590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-df5018de-4784-44c2-9acc-22634bf32e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:17.807587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acb9ff-e0c6-42bf-bb08-7d4ede168e94 service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:17.817190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:17.817962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:17.818305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:17.818664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:17.819402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:17.819830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:17.822062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:17.840825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 857/1711] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:17.842409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:17.845980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:17.863469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0d706-51ea-4d5a-88fa-43c8e0aed765 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:17.872861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:17.873693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:17.874072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:17.874494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] PATH_INFO: `/v3/users/10e2f645a24d40c78a15bc73d6f01c4c` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:17.876582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e2f645a24d40c78a15bc73d6f01c4c)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:17.878872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:17.975714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e2f645a24d40c78a15bc73d6f01c4c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:17.975817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 855/1712] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:17.976989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e2f645a24d40c78a15bc73d6f01c4c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:17.977797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:17.978149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:17.978525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:17.991132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e2f645a24d40c78a15bc73d6f01c4c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:18.009185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:18.011018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:18.034380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e2f645a24d40c78a15bc73d6f01c4c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:18.040302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 856/1713] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:18.041092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:18.044759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:18.054974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4b52564d-c15a-4543-8bbe-caa3e82ebd5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e2f645a24d40c78a15bc73d6f01c4c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:18.069424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 858/1714] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:17 2022] DELETE /identity/v3/users/10e2f645a24d40c78a15bc73d6f01c4c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:18.073016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8af0-464a-4917-b59d-d2e12fbe50cd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:18.073309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8af0-464a-4917-b59d-d2e12fbe50cd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:18.073588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec8af0-464a-4917-b59d-d2e12fbe50cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:18.094369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1785c9-e902-474d-b9ef-3d7395523e58 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:18.124195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-b3ec8af0-464a-4917-b59d-d2e12fbe50cd None None] MFA Rules not processed for user `b3098a7103d94b29bc4bce30cfd05bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:18.128323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b3ec8af0-464a-4917-b59d-d2e12fbe50cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:18.147346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:18.275655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:18.323727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:18.324009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:18.324255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:18.326276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:18.326961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:18.328986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:18.359557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 859/1715] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:18.361865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:18.362153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:18.362434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:18.407351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 857/1716] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:18 2022] POST /identity/v3/projects => generated 371 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:18.413513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:18.496954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] MFA Rules not processed for user `5e113c9ec968460ba070cfc1a69a25a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:18.501133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:18.626991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c229273-d756-41ec-b3ee-66a8fa15647a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:18.642136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:18.642837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:18.643295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:18.643657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:18.645806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:18.646161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:18.653724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:18.684484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 858/1717] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:18.686113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:18.717110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e1f08-ce13-45ea-8b30-0887105bbbe0 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:18.735462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:18.736325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:18.736902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:18.737740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:18.741812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:18.748967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:18.795005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 860/1718] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:18.796754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:18.798944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:18.832800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 859/1719] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:18 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:18.835354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:18.871172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f77504-6f42-45dd-ac17-62c62257a1c7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:18.887221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:18.888132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:18.888560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:18.890248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:18.891120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:18.897750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:18.939379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 860/1720] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:18 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:18.940525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.055315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28571b28-8945-4085-9342-de5b0ccb52fd None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:19.118819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7ddb4-e40f-4bbe-bfd7-28cce07c98ec admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:19.136005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:19.137034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.137338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.137704 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.139258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:19.144166 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:19.151075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:19.182383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 861/1721] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:19.183336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.209723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87529914-6374-410c-9d4a-69ed51a6e879 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:19.222589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2d1ce-9e00-43cb-b035-982d2f958c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:19.223233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d1ce-9e00-43cb-b035-982d2f958c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.223517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d1ce-9e00-43cb-b035-982d2f958c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.223792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d1ce-9e00-43cb-b035-982d2f958c83 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.224449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2d1ce-9e00-43cb-b035-982d2f958c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:19.226460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2d1ce-9e00-43cb-b035-982d2f958c83 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:19.232549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:19.233451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.233871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.234325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.238243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:19.261477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:19.276981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 862/1722] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:19 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:19.282581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4c0b6-bd76-47b0-9e70-7a99edf3a9b7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.283055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4c0b6-bd76-47b0-9e70-7a99edf3a9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.283371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4c0b6-bd76-47b0-9e70-7a99edf3a9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.425115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:19.428617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:19.447948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c5c4c0b6-bd76-47b0-9e70-7a99edf3a9b7 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:19.458005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c5c4c0b6-bd76-47b0-9e70-7a99edf3a9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:19.464131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 861/1723] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:19.483320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.489577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:19.556568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944903d0-f8d8-43f4-ac6d-662a1c146723 service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:19.587074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:19.588345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.588629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.588975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.644638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 863/1724] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 15:59:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:19.646324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4c0b6-bd76-47b0-9e70-7a99edf3a9b7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.685207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4c0b6-bd76-47b0-9e70-7a99edf3a9b7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:19.690677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:19.691526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.691879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.692215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.692976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:19.707894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:19.725435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 864/1725] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:19.727391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.742445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294472ad-8738-4e0f-bba9-e2b80c11533f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:19.750239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:19.751213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.751832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.752539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.753490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:19.758677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:19.774656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 865/1726] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:19.775753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.794924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7d856-f141-4953-aa44-87f1f41dafde admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:19.800947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:19.801910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:19.802073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:19.802322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] PATH_INFO: `/v3/projects/a36678835b4a4925a8278438f382c1c2/users/78f0f1cce6664c3894910230f27d5a02/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:19.805705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:19.807914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:19.828701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36678835b4a4925a8278438f382c1c2, user_id=78f0f1cce6664c3894910230f27d5a02, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:19.832466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:19.846563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 862/1727] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:19.862536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.864584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 866/1728] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:19 2022] PUT /identity/v3/projects/a36678835b4a4925a8278438f382c1c2/users/78f0f1cce6664c3894910230f27d5a02/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:19.868115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:19.875403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:19.897916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ee6fc-daab-49a0-873c-2d9034579177 service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:20.002782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe735a-4499-456f-b4ba-a729e400330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:20.004400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe735a-4499-456f-b4ba-a729e400330b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:20.005033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe735a-4499-456f-b4ba-a729e400330b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:20.005458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe735a-4499-456f-b4ba-a729e400330b admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:20.009422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe735a-4499-456f-b4ba-a729e400330b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:20.012051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe735a-4499-456f-b4ba-a729e400330b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:20.033671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 863/1729] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:20.037531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:20.037823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:20.038103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:20.057460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc796c3-99b7-4508-8d2b-87fb1e7df401 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:20.069763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:20.071042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:20.071526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:20.071873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] PATH_INFO: `/v3/projects/dd13bf032a064ffeb1a1d5e41e7519b5` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:20.078356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd13bf032a064ffeb1a1d5e41e7519b5)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:20.081763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:20.105727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:20.109844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] MFA Rules not processed for user `78f0f1cce6664c3894910230f27d5a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:20.111399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:20.157840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:20.218360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 864/1730] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:20.218360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:20.221464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:20.228296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd13bf032a064ffeb1a1d5e41e7519b5', 'request_id': 'req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:20.259218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd13bf032a064ffeb1a1d5e41e7519b5', 'request_id': 'req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:20.272698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd13bf032a064ffeb1a1d5e41e7519b5', 'request_id': 'req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:20.298087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] The token cache is being invalidate because project dd13bf032a064ffeb1a1d5e41e7519b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:20.298330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:20.299241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:20.301039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 867/1731] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:19 2022] DELETE /identity/v3/projects/dd13bf032a064ffeb1a1d5e41e7519b5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:20.302034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:20.307632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:20.409219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffce44b-9d81-4fe5-af91-20ffe62ee3de admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:20.424926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:20.428897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:20.429374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:20.429799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] PATH_INFO: `/v3/projects/e74c9bc3c5dd46e09d0e20f73cbb97af/users/e042913fc6324090b1d5c842b0f89de0/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:20.480615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74c9bc3c5dd46e09d0e20f73cbb97af, user_id=e042913fc6324090b1d5c842b0f89de0, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:20.486742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:20.495816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b064c-6251-4efd-8f02-e792b825f4e8 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:20.526683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 868/1732] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:20 2022] PUT /identity/v3/projects/e74c9bc3c5dd46e09d0e20f73cbb97af/users/e042913fc6324090b1d5c842b0f89de0/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:20.527684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:20.540908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:20.640778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0d4dd-a5f2-4a3d-b2a2-872aa1d36851 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:20.649924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d4dd-a5f2-4a3d-b2a2-872aa1d36851 service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:20.650330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d4dd-a5f2-4a3d-b2a2-872aa1d36851 service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:20.650894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d4dd-a5f2-4a3d-b2a2-872aa1d36851 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:20.655167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ebb0d4dd-a5f2-4a3d-b2a2-872aa1d36851 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:20.776818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd354ce-d1d2-4df4-a1a8-741ee68ed5b7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:20.790893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397d98-4cba-4f5d-8dee-35e46ce3e3bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022516f130ec4ad6a9b7c46b417a5690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:20.791596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397d98-4cba-4f5d-8dee-35e46ce3e3bd service glance] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:20.791810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397d98-4cba-4f5d-8dee-35e46ce3e3bd service glance] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:20.792078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397d98-4cba-4f5d-8dee-35e46ce3e3bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:20.799273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-13397d98-4cba-4f5d-8dee-35e46ce3e3bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:20.838905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:20.852707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0d4dd-a5f2-4a3d-b2a2-872aa1d36851 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:20.854543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0d4dd-a5f2-4a3d-b2a2-872aa1d36851 service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:20.859028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:20.881403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 865/1733] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 15:59:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:59:20.884502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec2317-a667-411f-91fb-6dd275db0dad None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:20.884502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec2317-a667-411f-91fb-6dd275db0dad None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:20.884502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec2317-a667-411f-91fb-6dd275db0dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:20.936279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-20ec2317-a667-411f-91fb-6dd275db0dad None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:20.941734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-20ec2317-a667-411f-91fb-6dd275db0dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:20.990682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13397d98-4cba-4f5d-8dee-35e46ce3e3bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:20.992534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13397d98-4cba-4f5d-8dee-35e46ce3e3bd service glance] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:21.022900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 869/1734] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:21.028276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef4fd-7c62-4f21-a41c-d85e689a44d0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:21.028564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef4fd-7c62-4f21-a41c-d85e689a44d0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:21.028873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef4fd-7c62-4f21-a41c-d85e689a44d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:21.098744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c8eef4fd-7c62-4f21-a41c-d85e689a44d0 None None] MFA Rules not processed for user `73a1aa0996db4c8ea851b2ba45026b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:21.099963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c8eef4fd-7c62-4f21-a41c-d85e689a44d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:21.156686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 866/1735] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:21.156686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa8625-b7df-409a-86e5-f5772157125e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:21.156686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa8625-b7df-409a-86e5-f5772157125e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:21.156686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa8625-b7df-409a-86e5-f5772157125e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:21.286471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-25aa8625-b7df-409a-86e5-f5772157125e None None] MFA Rules not processed for user `e042913fc6324090b1d5c842b0f89de0`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:21.292805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-25aa8625-b7df-409a-86e5-f5772157125e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:21.393475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 870/1736] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:21.394978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dda8-d635-4067-8ffe-23bb0827a147 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:21.395265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dda8-d635-4067-8ffe-23bb0827a147 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:21.395540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dda8-d635-4067-8ffe-23bb0827a147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:21.471664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-61f7dda8-d635-4067-8ffe-23bb0827a147 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:21.473539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-61f7dda8-d635-4067-8ffe-23bb0827a147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:21.571560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 871/1737] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:21.572547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7dda8-d635-4067-8ffe-23bb0827a147 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:21.625147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7dda8-d635-4067-8ffe-23bb0827a147 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:21.657051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:21.661635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:21.665114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:21.665903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:21.667876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:21.671140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:21.680755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:21.692504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 867/1738] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:21.694417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa8625-b7df-409a-86e5-f5772157125e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:21.744224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 872/1739] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:21.752529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:21.775391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa8625-b7df-409a-86e5-f5772157125e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:21.797176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9aeda-2cd2-4705-84a2-7ce92d16ec9f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:21.805251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:21.805909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:21.806186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:21.806455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:21.807107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:21.818591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:21.824972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:21.831297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:21.833077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:21.834574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:21.835407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:21.861543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:21.932572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 868/1740] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:21 2022] POST /identity/v3/projects => generated 365 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:21.933993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:21.944037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 873/1741] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:21 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:21.950134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:22.150685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7edf9-0fb6-4729-b88f-6d87966de477 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.163541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e26e8-3bc8-48a4-a8b3-e9759b14416d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.164617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:22.165424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.165674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.165950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.166600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:22.168578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:22.179524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:22.182848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.183413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.183968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.185199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:22.191208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:22.199132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:22.227024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 869/1742] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:21 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:22.230269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:22.255875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 874/1743] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:21 2022] POST /identity/v3/users => generated 283 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:22.261120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:22.262352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a54590-155b-4572-9c4c-831f2fa7463c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.283324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:22.284257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.286921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.287538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.288370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:22.292494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:22.294929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8eea-6cf7-4a05-87e3-8c607146d0e9 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.306035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:22.307325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.307673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.308015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.309382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:22.311249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:22.313796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:22.339483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 875/1744] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:59:22.351786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:22.366454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 870/1745] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:22.367680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:22.370350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed3353-e04e-4615-88f6-f65b1d98da50 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.383718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fcaf6-0875-4336-bca9-341a93907620 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.385925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:22.387541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.387824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.388153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.388911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:22.390277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:22.391905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.392392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.392923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] PATH_INFO: `/v3/projects/725968f875aa43c08da053a262cb7184/users/0fb913821c8849e8ba7f33621827d59c/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.394098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:22.418425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 876/1746] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:22.431877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:22.453593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725968f875aa43c08da053a262cb7184, user_id=0fb913821c8849e8ba7f33621827d59c, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:22.461136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd123675-3b21-4e3f-84e3-d5dbe13bc071 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.467214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d786bcc-f061-446f-b95d-03efef859fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:22.468445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:22.470635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bcc-f061-446f-b95d-03efef859fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.471513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bcc-f061-446f-b95d-03efef859fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.472407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bcc-f061-446f-b95d-03efef859fc2 admin admin] PATH_INFO: `/v3/projects/f0cb9b2881ac49e2b21d6a3349762e4c/users/49942c15532843ef88342495971f32cf/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.509712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 871/1747] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:22 2022] PUT /identity/v3/projects/725968f875aa43c08da053a262cb7184/users/0fb913821c8849e8ba7f33621827d59c/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:22.514284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:22.554205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d786bcc-f061-446f-b95d-03efef859fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cb9b2881ac49e2b21d6a3349762e4c, user_id=49942c15532843ef88342495971f32cf, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:22.557037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d786bcc-f061-446f-b95d-03efef859fc2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:22.606341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 877/1748] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:22 2022] PUT /identity/v3/projects/f0cb9b2881ac49e2b21d6a3349762e4c/users/49942c15532843ef88342495971f32cf/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:22.617383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce631ab-a96e-4437-af8d-149bfdd5eae5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.617985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce631ab-a96e-4437-af8d-149bfdd5eae5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.618479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce631ab-a96e-4437-af8d-149bfdd5eae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.732856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4ce631ab-a96e-4437-af8d-149bfdd5eae5 None None] MFA Rules not processed for user `49942c15532843ef88342495971f32cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:22.740897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4ce631ab-a96e-4437-af8d-149bfdd5eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:22.832095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc86633-09d8-45e3-a20c-5962fa82d1e0 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:22.879696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 878/1749] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:22.880662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe437d8-6dcf-41c2-a185-bd0dd9ee901f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:22.880662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe437d8-6dcf-41c2-a185-bd0dd9ee901f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:22.881452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe437d8-6dcf-41c2-a185-bd0dd9ee901f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:22.935151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-3fe437d8-6dcf-41c2-a185-bd0dd9ee901f None None] MFA Rules not processed for user `2d007a3d205345aebcc36a36d039da2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:22.939720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3fe437d8-6dcf-41c2-a185-bd0dd9ee901f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:23.018476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:23.022719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.024999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.025504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.028573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:23.036874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:23.080933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 872/1750] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:23.083607 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:23.122946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b9d4f-a413-499c-8361-3a0bb99a1dbe admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:23.134843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9b604-9172-4828-80b3-eb2cfcbacfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:23.135519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9b604-9172-4828-80b3-eb2cfcbacfba admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.135931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9b604-9172-4828-80b3-eb2cfcbacfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.136211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9b604-9172-4828-80b3-eb2cfcbacfba admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.150269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9b604-9172-4828-80b3-eb2cfcbacfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:23.161487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9b604-9172-4828-80b3-eb2cfcbacfba admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:23.164562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 879/1751] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Oct 3 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:23.166047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe437d8-6dcf-41c2-a185-bd0dd9ee901f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:23.180923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe437d8-6dcf-41c2-a185-bd0dd9ee901f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:23.186268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:23.198735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.199225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.199514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] PATH_INFO: `/v3/projects/725968f875aa43c08da053a262cb7184/users/0fb913821c8849e8ba7f33621827d59c/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.209748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] RBAC: Authorizing `identity:create_grant(project_id=725968f875aa43c08da053a262cb7184, user_id=0fb913821c8849e8ba7f33621827d59c, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:23.215520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:23.247382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 880/1752] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:23 2022] PUT /identity/v3/projects/725968f875aa43c08da053a262cb7184/users/0fb913821c8849e8ba7f33621827d59c/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:23.252883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:23.255017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 873/1753] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:23 2022] POST /identity/v3/projects => generated 367 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:23.258639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:23.261006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcba30-32f6-4274-a5db-c0aa5caee28f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.263814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcba30-32f6-4274-a5db-c0aa5caee28f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.263976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcba30-32f6-4274-a5db-c0aa5caee28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.341117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-3efcba30-32f6-4274-a5db-c0aa5caee28f None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:23.341540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:23.346648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3efcba30-32f6-4274-a5db-c0aa5caee28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:23.354584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c59a7-0084-4c4b-9190-fe12a4e6380f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:23.429120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cb477-ff24-4423-a72e-ec12551b126d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:23.430044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cb477-ff24-4423-a72e-ec12551b126d service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.430175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cb477-ff24-4423-a72e-ec12551b126d service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.430564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cb477-ff24-4423-a72e-ec12551b126d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.440129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cb477-ff24-4423-a72e-ec12551b126d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:23.441665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cb477-ff24-4423-a72e-ec12551b126d service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:23.453084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 881/1754] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:23.455170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef8e7-0505-464d-aefa-0d8dc62146e0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.455491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef8e7-0505-464d-aefa-0d8dc62146e0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.455764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef8e7-0505-464d-aefa-0d8dc62146e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.472103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:23.528641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 874/1755] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:23.529668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcba30-32f6-4274-a5db-c0aa5caee28f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:23.545492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-af2ef8e7-0505-464d-aefa-0d8dc62146e0 None None] MFA Rules not processed for user `0fb913821c8849e8ba7f33621827d59c`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:23.556648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-af2ef8e7-0505-464d-aefa-0d8dc62146e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:23.733453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcba30-32f6-4274-a5db-c0aa5caee28f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:23.734348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 882/1756] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:23.736237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763fedb-9494-4fac-8b63-9a4bc71f6796 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.737819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763fedb-9494-4fac-8b63-9a4bc71f6796 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.740240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763fedb-9494-4fac-8b63-9a4bc71f6796 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.750597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 883/1757] 10.222.0.33 () {54 vars in 912 bytes} [Mon Oct 3 15:59:23 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:59:23.752829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763fedb-9494-4fac-8b63-9a4bc71f6796 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:23.754233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:23.754890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:23.755170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:23.755463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:23.756078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:23.765453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:23.777250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6763fedb-9494-4fac-8b63-9a4bc71f6796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:23.824994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 875/1758] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:23 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:23.835665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:23.848811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:24.120570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946543d-46b7-4071-84b3-cc87edfa299c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:24.135250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88711810-8df6-4544-88c3-a0a85cf067ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:24.135893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88711810-8df6-4544-88c3-a0a85cf067ef service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.136173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88711810-8df6-4544-88c3-a0a85cf067ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.136446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88711810-8df6-4544-88c3-a0a85cf067ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:24.164450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88711810-8df6-4544-88c3-a0a85cf067ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:24.176006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88711810-8df6-4544-88c3-a0a85cf067ef service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:24.185066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763fedb-9494-4fac-8b63-9a4bc71f6796 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:24.210787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:24.211415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.211718 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.211956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:24.226705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 876/1759] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:24.227890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.228021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.232855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:24.272130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:24.275936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:24.349041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] MFA Rules not processed for user `2d007a3d205345aebcc36a36d039da2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:24.354163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:24.354367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 884/1760] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 15:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 15:59:24.363100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:24.386636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 877/1761] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:24.389572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:24.397717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:24.413856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d2663-4724-4a0b-a4b7-0f6a5f451ae2 service swift] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:24.423385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:24.436586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.439635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.440482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:24.441431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:24.445210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:24.455616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331e811-d22d-463c-b946-b2636ce509f2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:24.480610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:24.481198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.485529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.489135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] PATH_INFO: `/v3/users/aaafa02a240c4d389288dcfc7aefb471` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:24.491720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaafa02a240c4d389288dcfc7aefb471)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:24.495169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 885/1762] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:24.502469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:24.504181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:24.525787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:24.632122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c426d3-d079-4577-a170-21cfd0b9b422 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:24.673874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:24.674777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.675981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.684798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:24.710913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:24.722389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:24.731617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:24.786691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 886/1763] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:24.788700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:24.796862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaafa02a240c4d389288dcfc7aefb471'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:24.805243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaafa02a240c4d389288dcfc7aefb471'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:24.838006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaafa02a240c4d389288dcfc7aefb471'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:24.852470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37d75b-c570-4712-8a71-7d6d80d903a0 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:24.893749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929ee52-5eb7-4415-a367-37f72656f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:24.896103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ee52-5eb7-4415-a367-37f72656f34f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.898409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ee52-5eb7-4415-a367-37f72656f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.901328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929ee52-5eb7-4415-a367-37f72656f34f admin admin] PATH_INFO: `/v3/projects/e7596cf5d5fd4bbeaa09af0be05f53b5/users/b09cb3acaeab4941883a8b25e2fdcbef/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:24.907991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:24.915929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaafa02a240c4d389288dcfc7aefb471'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:24.953098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fdf51fd7-d15c-4b4b-9b2d-c9eb74687b69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaafa02a240c4d389288dcfc7aefb471'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:24.966676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929ee52-5eb7-4415-a367-37f72656f34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7596cf5d5fd4bbeaa09af0be05f53b5, user_id=b09cb3acaeab4941883a8b25e2fdcbef, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:24.973582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 878/1764] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:24 2022] DELETE /identity/v3/users/aaafa02a240c4d389288dcfc7aefb471 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:24.975282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929ee52-5eb7-4415-a367-37f72656f34f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:24.989265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:24.994524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:24.994953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:25.030018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 887/1765] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:24 2022] PUT /identity/v3/projects/e7596cf5d5fd4bbeaa09af0be05f53b5/users/b09cb3acaeab4941883a8b25e2fdcbef/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:25.036486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:25.042171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:25.042171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:25.051230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] MFA Rules not processed for user `b3098a7103d94b29bc4bce30cfd05bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:25.056379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:25.132921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:25.135881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:25.274859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 879/1766] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:25.285994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:25.290721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:25.480691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 888/1767] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:25.497002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:25.501592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:25.532260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaeb10-cfeb-44ff-ab13-3abe712221f1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:25.548878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:25.549514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:25.549794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:25.550064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:25.550980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:25.552805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:25.580507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 880/1768] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:25.596227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:25.648701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adea4fd-6943-482d-b86f-7c73596eaa00 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:25.700130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde4146-d30f-44d1-ba96-84f5a1f61d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:25.719654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4146-d30f-44d1-ba96-84f5a1f61d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:25.721349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4146-d30f-44d1-ba96-84f5a1f61d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:25.721913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4146-d30f-44d1-ba96-84f5a1f61d01 admin admin] PATH_INFO: `/v3/projects/e7596cf5d5fd4bbeaa09af0be05f53b5/users/b09cb3acaeab4941883a8b25e2fdcbef/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:25.740920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde4146-d30f-44d1-ba96-84f5a1f61d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7596cf5d5fd4bbeaa09af0be05f53b5, user_id=b09cb3acaeab4941883a8b25e2fdcbef, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:25.769798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde4146-d30f-44d1-ba96-84f5a1f61d01 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:25.774914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34307e49-e7e4-44df-8813-f8725900ff13 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:25.790061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:25.791677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:25.791978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:25.793639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:25.833815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 881/1769] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:25 2022] PUT /identity/v3/projects/e7596cf5d5fd4bbeaa09af0be05f53b5/users/b09cb3acaeab4941883a8b25e2fdcbef/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:25.834048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:25.834048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:25.844342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36d08-8713-49aa-9e78-f4689309d64f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:25.844736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36d08-8713-49aa-9e78-f4689309d64f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:25.845340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36d08-8713-49aa-9e78-f4689309d64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:25.871877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 889/1770] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:25.947174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2fe36d08-8713-49aa-9e78-f4689309d64f None None] MFA Rules not processed for user `b09cb3acaeab4941883a8b25e2fdcbef`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:25.950883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2fe36d08-8713-49aa-9e78-f4689309d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:26.264129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:26.292479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 882/1771] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:26.302634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:26.304172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:26.567383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dcf63-281d-46d1-bc50-849db7fb595f service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:26.667337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:26.732290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e9023-8bfd-4cb3-aec5-45f52b1d361e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:26.735300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9023-8bfd-4cb3-aec5-45f52b1d361e service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:26.735558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9023-8bfd-4cb3-aec5-45f52b1d361e service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:26.736894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e9023-8bfd-4cb3-aec5-45f52b1d361e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:26.769541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e9023-8bfd-4cb3-aec5-45f52b1d361e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:26.773018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e9023-8bfd-4cb3-aec5-45f52b1d361e service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:26.804316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 890/1772] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:27.042544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe36d08-8713-49aa-9e78-f4689309d64f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:27.068485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe36d08-8713-49aa-9e78-f4689309d64f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:27.097662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:27.099227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:27.103971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:27.105877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:27.118403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:27.450279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:27.458124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:27.503414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 883/1773] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:27.519099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd82c1c-c611-4728-9490-f5b294bf0a93 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:27.519369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd82c1c-c611-4728-9490-f5b294bf0a93 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:27.519605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd82c1c-c611-4728-9490-f5b294bf0a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:27.608029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bbd82c1c-c611-4728-9490-f5b294bf0a93 None None] MFA Rules not processed for user `5e113c9ec968460ba070cfc1a69a25a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:27.613186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bbd82c1c-c611-4728-9490-f5b294bf0a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:27.891549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 891/1774] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:28.099875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:28.102206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:28.385473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2fe2-fc2d-44c7-b0cf-0f2aca787ca1 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:28.385944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2fe2-fc2d-44c7-b0cf-0f2aca787ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:28.386355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2fe2-fc2d-44c7-b0cf-0f2aca787ca1 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:28.388383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 892/1775] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 15:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:59:28.491703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2fe2-fc2d-44c7-b0cf-0f2aca787ca1 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:28.496230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bb3e2fe2-fc2d-44c7-b0cf-0f2aca787ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:28.614756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b778-e96b-4ace-8940-9faae20bd339 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:28.630873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:28.631611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:28.631960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:28.641452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] PATH_INFO: `/v3/projects/ca133ddf3c61490093232365a8555999` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:28.645434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca133ddf3c61490093232365a8555999)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:28.650118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:28.756668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2fe2-fc2d-44c7-b0cf-0f2aca787ca1 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:28.773489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:28.775468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:28.777564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:28.779557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] PATH_INFO: `/v3/users/e042913fc6324090b1d5c842b0f89de0` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:28.784849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e042913fc6324090b1d5c842b0f89de0)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:28.791096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:28.839304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca133ddf3c61490093232365a8555999', 'request_id': 'req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:28.860654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca133ddf3c61490093232365a8555999', 'request_id': 'req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:28.881089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca133ddf3c61490093232365a8555999', 'request_id': 'req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:28.921437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] The token cache is being invalidate because project ca133ddf3c61490093232365a8555999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:28.921677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:28.925272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d7844ebf-5ad0-4bd9-8b6e-5caa1f655e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:28.927314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 884/1776] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:28 2022] DELETE /identity/v3/projects/ca133ddf3c61490093232365a8555999 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:59:28.932578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb946a-9907-443f-9229-aa62f23ac247 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:28.932980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb946a-9907-443f-9229-aa62f23ac247 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:28.933361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb946a-9907-443f-9229-aa62f23ac247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:28.941115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e042913fc6324090b1d5c842b0f89de0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:28.963541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e042913fc6324090b1d5c842b0f89de0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:28.979327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-addb946a-9907-443f-9229-aa62f23ac247 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:28.982465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-addb946a-9907-443f-9229-aa62f23ac247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:29.011112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e042913fc6324090b1d5c842b0f89de0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:29.041006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:29.057749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e042913fc6324090b1d5c842b0f89de0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:29.058918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fed739f5-204d-4b21-a732-321d23de4e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e042913fc6324090b1d5c842b0f89de0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:29.062452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:29.084798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 893/1777] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:28 2022] DELETE /identity/v3/users/e042913fc6324090b1d5c842b0f89de0 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:59:29.088387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916e97-6624-4a2c-a29d-bc74c33d2112 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:29.090158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916e97-6624-4a2c-a29d-bc74c33d2112 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:29.091605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916e97-6624-4a2c-a29d-bc74c33d2112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:29.139665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 885/1778] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:29.141870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:29.146323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:29.147060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:29.153564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ec916e97-6624-4a2c-a29d-bc74c33d2112 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:29.155615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ec916e97-6624-4a2c-a29d-bc74c33d2112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:29.216556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:29.231165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:29.402660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 894/1779] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:29.404334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa53d4-aefc-4fe7-9b5e-cf0122b10521 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:29.406225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa53d4-aefc-4fe7-9b5e-cf0122b10521 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:29.406669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa53d4-aefc-4fe7-9b5e-cf0122b10521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:29.441655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 886/1780] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:29.459921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:29.466048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:29.471355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-18aa53d4-aefc-4fe7-9b5e-cf0122b10521 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:29.472648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-18aa53d4-aefc-4fe7-9b5e-cf0122b10521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:29.494580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 895/1781] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:29.499355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa53d4-aefc-4fe7-9b5e-cf0122b10521 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:29.533521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa53d4-aefc-4fe7-9b5e-cf0122b10521 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:29.557184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:29.558089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:29.558705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:29.561844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:29.563361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:29.565360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:29.567981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:29.645873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 896/1782] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:29.650639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:29.678730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e91aa1-ee2d-4a7c-81c7-d1a711b82c2b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:29.681093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726286-d854-4837-9781-668ee4b463eb admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:29.683282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:29.684215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:29.684596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:29.684994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:29.691605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:29.692478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:29.692928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:29.693879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:29.694819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:29.696782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:29.697580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:29.699342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:29.735548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 887/1783] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:29.783639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 897/1784] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:29 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:29.784683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:29.799607 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70038bf-6f0a-4955-bf78-d823076f212d service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:29.967101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:29.967864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:29.968449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:29.969428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:29.970257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:29.972490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:30.018803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:30.034299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 888/1785] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:29 2022] POST /identity/v3/users => generated 282 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:30.035492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:30.052515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0f67d-e3ae-4ccd-993c-998f89864940 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:30.078623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:30.084947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:30.086741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:30.087023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:30.087713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:30.093217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:30.134249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:30.143276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 898/1786] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:59:30.148472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ebc95-c2d3-4dc5-b492-fdcc0e6b2116 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:30.179378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb53fb-7dea-4e12-888d-4215f4e5b700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:30.180196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb53fb-7dea-4e12-888d-4215f4e5b700 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:30.181400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb53fb-7dea-4e12-888d-4215f4e5b700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:30.181685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb53fb-7dea-4e12-888d-4215f4e5b700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:30.182563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb53fb-7dea-4e12-888d-4215f4e5b700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:30.184395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb53fb-7dea-4e12-888d-4215f4e5b700 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:30.240103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 889/1787] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:30.244620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:30.274837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57c01f-2670-4fbe-a284-aeec2b3fb99e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:30.289723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:30.293339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:30.293646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:30.293928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] PATH_INFO: `/v3/projects/e00962e815cd4ec2a90135d306bde922/users/ba783519c48549cfa09c43e8366592ce/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:30.340503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00962e815cd4ec2a90135d306bde922, user_id=ba783519c48549cfa09c43e8366592ce, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:30.348997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:30.381929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 899/1788] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:30 2022] PUT /identity/v3/projects/e00962e815cd4ec2a90135d306bde922/users/ba783519c48549cfa09c43e8366592ce/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:30.393707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1a23a-70e0-4c77-b845-272f33eb2d31 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:30.393993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1a23a-70e0-4c77-b845-272f33eb2d31 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:30.394265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1a23a-70e0-4c77-b845-272f33eb2d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:30.478801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-67d1a23a-70e0-4c77-b845-272f33eb2d31 None None] MFA Rules not processed for user `ba783519c48549cfa09c43e8366592ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:30.480126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-67d1a23a-70e0-4c77-b845-272f33eb2d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:30.658889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:30.665393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:30.842668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 890/1789] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:30.849042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1a23a-70e0-4c77-b845-272f33eb2d31 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:30.886717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7764f-fb93-4e36-976d-e1e52b15da4f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:30.902401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:30.903066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:30.903354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:30.903662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] PATH_INFO: `/v3/projects/e74c9bc3c5dd46e09d0e20f73cbb97af` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:30.905925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74c9bc3c5dd46e09d0e20f73cbb97af)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:30.908151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:30.955382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:31.043339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74c9bc3c5dd46e09d0e20f73cbb97af', 'request_id': 'req-16925f72-0928-4872-aec2-7e34584ffa1e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.061171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74c9bc3c5dd46e09d0e20f73cbb97af', 'request_id': 'req-16925f72-0928-4872-aec2-7e34584ffa1e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.065425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1a23a-70e0-4c77-b845-272f33eb2d31 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:31.077710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74c9bc3c5dd46e09d0e20f73cbb97af', 'request_id': 'req-16925f72-0928-4872-aec2-7e34584ffa1e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.101522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] The token cache is being invalidate because project e74c9bc3c5dd46e09d0e20f73cbb97af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:31.102137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.102980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.105610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 900/1790] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:30 2022] DELETE /identity/v3/projects/e74c9bc3c5dd46e09d0e20f73cbb97af => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:31.110036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:31.112201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:31.275905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:31.284524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:31.285449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:31.285892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:31.286899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:31.293225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:31.311746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16925f72-0928-4872-aec2-7e34584ffa1e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:31.355644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:31.356611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:31.362565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:31.363281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] PATH_INFO: `/v3/users/0fb913821c8849e8ba7f33621827d59c` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:31.366716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb913821c8849e8ba7f33621827d59c)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:31.374314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:31.376459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 891/1791] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:30 2022] POST /identity/v3/projects => generated 365 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:31.380615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:31.382886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:31.418334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:31.526040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb913821c8849e8ba7f33621827d59c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.548063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb913821c8849e8ba7f33621827d59c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.575366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb913821c8849e8ba7f33621827d59c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.596344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb913821c8849e8ba7f33621827d59c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.597133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb913821c8849e8ba7f33621827d59c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:31.603547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a432-d35c-4f91-8419-243c3b4f0f94 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:31.611647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 901/1792] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:31 2022] DELETE /identity/v3/users/0fb913821c8849e8ba7f33621827d59c => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:31.637570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:31.639969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:31.640259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:31.640533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:31.651824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:31.665498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:31.720767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 892/1793] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:31 2022] POST /identity/v3/users => generated 289 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:31.723008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:31.724792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:31.802062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:31.911596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1ffd0-72c3-4b1f-9801-9b2aa5dff223 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:31.936592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:31.937515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:31.938595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:31.939098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:31.939938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:31.942288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:31.957479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 902/1794] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:31.961709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:31.974628 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe3427-a073-4183-b9f4-a7c858d36da0 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:31.982604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:31.983339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:31.983757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:31.984286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] PATH_INFO: `/v3/projects/9a299ee7d37c40098bfadb5e92ad471f/users/cbf175cb56c84d288bcc66b65928cd4a/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:32.025532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a299ee7d37c40098bfadb5e92ad471f, user_id=cbf175cb56c84d288bcc66b65928cd4a, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:32.029483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:32.066489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 893/1795] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:31 2022] PUT /identity/v3/projects/9a299ee7d37c40098bfadb5e92ad471f/users/cbf175cb56c84d288bcc66b65928cd4a/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:32.070690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:32.091664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd3542-f38c-4891-90c1-1a3af4938d16 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:32.262301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d5317-7a7a-41a0-8d3d-b47248f1ab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:32.263980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d5317-7a7a-41a0-8d3d-b47248f1ab1c admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:32.264637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d5317-7a7a-41a0-8d3d-b47248f1ab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:32.265301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d5317-7a7a-41a0-8d3d-b47248f1ab1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:32.266800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d5317-7a7a-41a0-8d3d-b47248f1ab1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:32.269147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d5317-7a7a-41a0-8d3d-b47248f1ab1c admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:32.288689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 903/1796] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:32.292617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:32.333554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f90c3-f55f-47c9-a955-6b4df78f1c8d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:32.348609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:32.349974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:32.350503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:32.351012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] PATH_INFO: `/v3/projects/9a299ee7d37c40098bfadb5e92ad471f/users/cbf175cb56c84d288bcc66b65928cd4a/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:32.354198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a299ee7d37c40098bfadb5e92ad471f, user_id=cbf175cb56c84d288bcc66b65928cd4a, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:32.358046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:32.391534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 894/1797] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:32 2022] PUT /identity/v3/projects/9a299ee7d37c40098bfadb5e92ad471f/users/cbf175cb56c84d288bcc66b65928cd4a/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:32.405795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:32.407059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:32.408175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:32.470985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] MFA Rules not processed for user `cbf175cb56c84d288bcc66b65928cd4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:32.472231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:32.683674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 904/1798] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:32.707594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:32.709451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:32.839106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a6839-db0f-4c2f-86a3-8464faa877d7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:32.845210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:32.846104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:32.847286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:32.847578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:32.865483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:32.867211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:32.896772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 895/1799] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:32.937368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:32.939678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:33.097187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:33.170613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3444a-ab7f-4170-8f1e-8ce04be46e61 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:33.185526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:33.186096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:33.196192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:33.196507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] PATH_INFO: `/v3/projects/725968f875aa43c08da053a262cb7184` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:33.200056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=725968f875aa43c08da053a262cb7184)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:33.203107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:33.206490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:33.207947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:33.229246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32265263-0db0-467f-88ac-8aa63f114cc5 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:33.233933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e315acf-d547-4ee1-9851-2148ecec0650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:33.235470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315acf-d547-4ee1-9851-2148ecec0650 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:33.235961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315acf-d547-4ee1-9851-2148ecec0650 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:33.237221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e315acf-d547-4ee1-9851-2148ecec0650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:33.239697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8e315acf-d547-4ee1-9851-2148ecec0650 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:33.421587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725968f875aa43c08da053a262cb7184', 'request_id': 'req-25fe7348-c0c4-4165-a008-771de5b2f391'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:33.442311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725968f875aa43c08da053a262cb7184', 'request_id': 'req-25fe7348-c0c4-4165-a008-771de5b2f391'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:33.457536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e315acf-d547-4ee1-9851-2148ecec0650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:33.459492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e315acf-d547-4ee1-9851-2148ecec0650 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:33.466950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725968f875aa43c08da053a262cb7184', 'request_id': 'req-25fe7348-c0c4-4165-a008-771de5b2f391'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:33.491777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] The token cache is being invalidate because project 725968f875aa43c08da053a262cb7184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:33.492112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:33.493527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 896/1800] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:33.503676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25fe7348-c0c4-4165-a008-771de5b2f391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:33.513637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 905/1801] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:32 2022] DELETE /identity/v3/projects/725968f875aa43c08da053a262cb7184 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:33.518324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a4bba-909d-49e5-bcf3-e76d6e58ff82 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:33.519304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a4bba-909d-49e5-bcf3-e76d6e58ff82 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:33.519816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a4bba-909d-49e5-bcf3-e76d6e58ff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:33.537029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:33.568145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d38a4bba-909d-49e5-bcf3-e76d6e58ff82 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:33.570566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d38a4bba-909d-49e5-bcf3-e76d6e58ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:33.784933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 897/1802] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:33.789210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4498c6-88ea-45af-b754-eb722d2271b4 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:33.789497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4498c6-88ea-45af-b754-eb722d2271b4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:33.789824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4498c6-88ea-45af-b754-eb722d2271b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:33.848083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-da4498c6-88ea-45af-b754-eb722d2271b4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:33.850868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-da4498c6-88ea-45af-b754-eb722d2271b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:33.877829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 906/1803] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:33.879705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a4bba-909d-49e5-bcf3-e76d6e58ff82 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:33.906216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a4bba-909d-49e5-bcf3-e76d6e58ff82 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:33.927671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:33.929821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:33.931081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:33.935787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:33.940325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:33.940910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:33.945450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:33.992141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4498c6-88ea-45af-b754-eb722d2271b4 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:34.012299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 898/1804] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:59:34.019598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4498c6-88ea-45af-b754-eb722d2271b4 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:34.033797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:34.050326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:34.051997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:34.056372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:34.056372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:34.064552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:34.128050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:34.130637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 907/1805] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:33 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:34.181111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7904-1629-47f2-ba68-6f17ef7a277b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:34.306443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:34.307050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:34.307296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:34.307569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:34.308344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:34.319989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:34.364478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 899/1806] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:34 2022] POST /identity/v3/users => generated 288 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:34.369538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:34.394348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf847a3-40c2-4621-a48b-b793d09539c7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:34.417537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:34.418713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:34.418713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:34.420214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:34.421780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:34.430661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:34.461595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 908/1807] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:34.465901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:34.482294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f78fd-e48a-4b77-9114-3357ad653041 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:34.490686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919523c4-329b-4fd7-8006-ac54e3a3cea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:34.491655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919523c4-329b-4fd7-8006-ac54e3a3cea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:34.492369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919523c4-329b-4fd7-8006-ac54e3a3cea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:34.492914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919523c4-329b-4fd7-8006-ac54e3a3cea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:34.493938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919523c4-329b-4fd7-8006-ac54e3a3cea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:34.496024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919523c4-329b-4fd7-8006-ac54e3a3cea2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:34.521749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 900/1808] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:34.525421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:34.543015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4f4c4-1b97-42bf-afb6-9a341a58c26e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:34.556155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:34.557099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:34.557531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:34.558034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] PATH_INFO: `/v3/projects/6dcad8c29dc04887a3174a874bd7c023/users/6ac82d34dc09412593fc3940d873a2c7/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:34.580011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dcad8c29dc04887a3174a874bd7c023, user_id=6ac82d34dc09412593fc3940d873a2c7, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:34.583050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:34.605752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 909/1809] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:34 2022] PUT /identity/v3/projects/6dcad8c29dc04887a3174a874bd7c023/users/6ac82d34dc09412593fc3940d873a2c7/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:34.618601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699361bb-5da1-42f6-8783-50d348d7225e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:34.619044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699361bb-5da1-42f6-8783-50d348d7225e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:34.619194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699361bb-5da1-42f6-8783-50d348d7225e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:34.682538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-699361bb-5da1-42f6-8783-50d348d7225e None None] MFA Rules not processed for user `6ac82d34dc09412593fc3940d873a2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:34.684188 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-699361bb-5da1-42f6-8783-50d348d7225e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:34.857850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:34.863681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 901/1810] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:34.869547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.077121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62da54-cb09-47f4-954e-0bf91631bfd2 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.087305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aed1a3-00ed-4b92-85a2-939b504d6fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:35.088170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed1a3-00ed-4b92-85a2-939b504d6fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.088583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed1a3-00ed-4b92-85a2-939b504d6fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.088894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aed1a3-00ed-4b92-85a2-939b504d6fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.089662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aed1a3-00ed-4b92-85a2-939b504d6fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:35.092155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aed1a3-00ed-4b92-85a2-939b504d6fc2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:35.131159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:35.170704 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 910/1811] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:34 2022] POST /identity/v3/projects => generated 375 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:59:35.179394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699361bb-5da1-42f6-8783-50d348d7225e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.201561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699361bb-5da1-42f6-8783-50d348d7225e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.351604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8240-c285-4053-885e-1f3fe08f99b3 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.351899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8240-c285-4053-885e-1f3fe08f99b3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.352209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb8240-c285-4053-885e-1f3fe08f99b3 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.353572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 911/1812] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 15:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 15:59:35.394050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32715ed-9f28-4619-8fa6-da80cb19a849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:35.395469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32715ed-9f28-4619-8fa6-da80cb19a849 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.395897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32715ed-9f28-4619-8fa6-da80cb19a849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.396410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32715ed-9f28-4619-8fa6-da80cb19a849 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.397658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32715ed-9f28-4619-8fa6-da80cb19a849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:35.400614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32715ed-9f28-4619-8fa6-da80cb19a849 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:35.453854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 902/1813] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:35 2022] POST /identity/v3/users => generated 294 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:35.457679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8240-c285-4053-885e-1f3fe08f99b3 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.466899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf7d78-953f-451a-a602-b0bb6d62e513 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.467299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf7d78-953f-451a-a602-b0bb6d62e513 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.467650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf7d78-953f-451a-a602-b0bb6d62e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.484863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb8240-c285-4053-885e-1f3fe08f99b3 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.502922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:35.503543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.504026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.504146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.504828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:35.506439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:35.511998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-4dcf7d78-953f-451a-a602-b0bb6d62e513 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:35.513612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-4dcf7d78-953f-451a-a602-b0bb6d62e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:35.542464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 912/1814] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:35.544076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.546721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:35.563498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51735c71-0bf3-4c42-9c9f-27bff3490f7d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.572402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:35.574040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.574345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.575594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] PATH_INFO: `/v3/users/49942c15532843ef88342495971f32cf` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.576053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49942c15532843ef88342495971f32cf)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:35.578854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:35.689765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 903/1815] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 15:59:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:35.699962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf7d78-953f-451a-a602-b0bb6d62e513 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.716921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49942c15532843ef88342495971f32cf'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:35.722673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf7d78-953f-451a-a602-b0bb6d62e513 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.730918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49942c15532843ef88342495971f32cf'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:35.733890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:35.735258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.735591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.736126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] PATH_INFO: `/v3/projects/603b0b72733d4ccfbf98b75be9610a46/users/ad79c4ff515e4af2a610d89945f1ab4b/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.752408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49942c15532843ef88342495971f32cf'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:35.756321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=603b0b72733d4ccfbf98b75be9610a46, user_id=ad79c4ff515e4af2a610d89945f1ab4b, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:35.759080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:35.762608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49942c15532843ef88342495971f32cf'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:35.763533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0c8abfeb-e677-4e2d-ab02-6aee89ad3ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49942c15532843ef88342495971f32cf'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:35.770828 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 913/1816] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:35 2022] DELETE /identity/v3/users/49942c15532843ef88342495971f32cf => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:35.775702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.776195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.776336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.777146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 904/1817] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:35 2022] PUT /identity/v3/projects/603b0b72733d4ccfbf98b75be9610a46/users/ad79c4ff515e4af2a610d89945f1ab4b/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:35.780759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.783426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:35.799974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:35.801231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:35.869861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:35.875477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8fc36-6c96-491b-9e92-d2aeba60b280 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.879183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 914/1818] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:35.885678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.887208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:35.889429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:35.890130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.890413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.890864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.891835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:35.894460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:35.911867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 905/1819] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:35.913384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:35.924360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1e4d-d3e2-43d3-aff2-2badc6f383ff admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.929775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9042b382-761c-4a0d-b014-a501d5bbd10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:35.930427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042b382-761c-4a0d-b014-a501d5bbd10a admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.930732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042b382-761c-4a0d-b014-a501d5bbd10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.931006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9042b382-761c-4a0d-b014-a501d5bbd10a admin admin] PATH_INFO: `/v3/projects/603b0b72733d4ccfbf98b75be9610a46/users/ad79c4ff515e4af2a610d89945f1ab4b/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.933160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042b382-761c-4a0d-b014-a501d5bbd10a admin admin] RBAC: Authorizing `identity:create_grant(project_id=603b0b72733d4ccfbf98b75be9610a46, user_id=ad79c4ff515e4af2a610d89945f1ab4b, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:35.936004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9042b382-761c-4a0d-b014-a501d5bbd10a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:35.956532 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 906/1820] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:35 2022] PUT /identity/v3/projects/603b0b72733d4ccfbf98b75be9610a46/users/ad79c4ff515e4af2a610d89945f1ab4b/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:35.962577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed6dc7-1050-494b-b73f-eb9c64683f0c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:35.967496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:35.967776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:35.968310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:35.973110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:36.009167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] MFA Rules not processed for user `ad79c4ff515e4af2a610d89945f1ab4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:36.010626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:36.034336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:36.035700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:36.035700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:36.035700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:36.100522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 907/1821] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:36.112610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:36.114270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:36.119882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:36.121696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:36.134802 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 915/1822] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:36.188770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44409300-8ba7-4e1b-b1fa-2149b17042bb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:36.193399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:36.194024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:36.194299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:36.194568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:36.196885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:36.275162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:36.276656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:36.291519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 908/1823] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:36.452545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:36.454860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:36.472091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dbe73-b460-4ac1-aad5-99f4f46a2a7e service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:36.496342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:36.500991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:36.501311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:36.501618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] PATH_INFO: `/v3/projects/f0cb9b2881ac49e2b21d6a3349762e4c` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:36.506880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cb9b2881ac49e2b21d6a3349762e4c)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:36.509500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:36.595331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cb9b2881ac49e2b21d6a3349762e4c', 'request_id': 'req-b58d5c24-271c-4bca-be32-271336c4842d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.605841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cb9b2881ac49e2b21d6a3349762e4c', 'request_id': 'req-b58d5c24-271c-4bca-be32-271336c4842d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.617814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cb9b2881ac49e2b21d6a3349762e4c', 'request_id': 'req-b58d5c24-271c-4bca-be32-271336c4842d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.632477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] The token cache is being invalidate because project f0cb9b2881ac49e2b21d6a3349762e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:36.632952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.633810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.638636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 916/1824] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:36 2022] DELETE /identity/v3/projects/f0cb9b2881ac49e2b21d6a3349762e4c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:36.639862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:36.660664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8637a-7143-4dd4-b230-8df4df1b81a7 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:36.692480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:36.772170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:36.773256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:36.773529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:36.773791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] PATH_INFO: `/v3/users/b09cb3acaeab4941883a8b25e2fdcbef` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:36.774992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09cb3acaeab4941883a8b25e2fdcbef)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:36.777046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:36.849394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09cb3acaeab4941883a8b25e2fdcbef'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.850453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09cb3acaeab4941883a8b25e2fdcbef'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.864213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09cb3acaeab4941883a8b25e2fdcbef'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.894010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09cb3acaeab4941883a8b25e2fdcbef'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.904759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09cb3acaeab4941883a8b25e2fdcbef'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:36.912606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 909/1825] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:36 2022] DELETE /identity/v3/users/b09cb3acaeab4941883a8b25e2fdcbef => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:36.917313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:36.918897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:36.935194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d5c24-271c-4bca-be32-271336c4842d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:36.999789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:37.051406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:37.052411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:37.052957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:37.053649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] PATH_INFO: `/v3/users/ba783519c48549cfa09c43e8366592ce` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:37.055518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba783519c48549cfa09c43e8366592ce)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:37.064922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:37.152691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba783519c48549cfa09c43e8366592ce'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.169805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba783519c48549cfa09c43e8366592ce'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.197287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba783519c48549cfa09c43e8366592ce'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.215198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba783519c48549cfa09c43e8366592ce'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.216351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a9eed4ec-eeaf-44da-8a00-f7f53c69ee36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba783519c48549cfa09c43e8366592ce'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.232263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 917/1826] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:36 2022] DELETE /identity/v3/users/ba783519c48549cfa09c43e8366592ce => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:37.234121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:37.236873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:37.239312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:37.240099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:37.242719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:37.273306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:37.274749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:37.411931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c222365-ab9b-48a2-bcac-4ef2e9537949 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:37.464854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 918/1827] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:37.466741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:37.477765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:37.502937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:37.579816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:37.580588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:37.580969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:37.581240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:37.583862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:37.600075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:37.602015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:37.605490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:37.623520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 910/1828] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:37.625569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:37.628066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:37.633105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31a8d9-f93f-4ff4-8680-e44fec18c3a7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:37.648501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:37.649285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:37.649637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:37.650060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:37.652771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:37.681686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:37.684005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:37.717719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 919/1829] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:37.720204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:37.724782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:37.778608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29006b6-816a-4c83-b3aa-e1ebe5a66b30 service nova] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:37.785730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edd950-b6ab-4738-b82f-f30b26edd8fb service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:37.790653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:37.792458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:37.792998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:37.793875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] PATH_INFO: `/v3/projects/e7596cf5d5fd4bbeaa09af0be05f53b5` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:37.798315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7596cf5d5fd4bbeaa09af0be05f53b5)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:37.799038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3183fc0-f6b8-44f2-a9c8-fe2c5f488674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:37.799855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3183fc0-f6b8-44f2-a9c8-fe2c5f488674 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:37.800326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3183fc0-f6b8-44f2-a9c8-fe2c5f488674 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:37.800821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3183fc0-f6b8-44f2-a9c8-fe2c5f488674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:37.805990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:37.825972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3183fc0-f6b8-44f2-a9c8-fe2c5f488674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:37.827687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3183fc0-f6b8-44f2-a9c8-fe2c5f488674 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:37.847889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 911/1830] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:37.933503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7596cf5d5fd4bbeaa09af0be05f53b5', 'request_id': 'req-cc7673a8-12e2-4049-94fe-80400f1c2075'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.945519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7596cf5d5fd4bbeaa09af0be05f53b5', 'request_id': 'req-cc7673a8-12e2-4049-94fe-80400f1c2075'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.954766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7596cf5d5fd4bbeaa09af0be05f53b5', 'request_id': 'req-cc7673a8-12e2-4049-94fe-80400f1c2075'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.968178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] The token cache is being invalidate because project e7596cf5d5fd4bbeaa09af0be05f53b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:37.968901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.970408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cc7673a8-12e2-4049-94fe-80400f1c2075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:37.971431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 920/1831] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:37 2022] DELETE /identity/v3/projects/e7596cf5d5fd4bbeaa09af0be05f53b5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:37.979581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77004b3a-67a7-400c-8653-cc504292bcff None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:37.980146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77004b3a-67a7-400c-8653-cc504292bcff None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:37.980584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77004b3a-67a7-400c-8653-cc504292bcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.010262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-77004b3a-67a7-400c-8653-cc504292bcff None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:38.011485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-77004b3a-67a7-400c-8653-cc504292bcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:38.178717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 912/1832] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:38.186992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06870e-74e4-432f-9c5d-f27b0108d968 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.188064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06870e-74e4-432f-9c5d-f27b0108d968 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.188345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06870e-74e4-432f-9c5d-f27b0108d968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.222823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-5e06870e-74e4-432f-9c5d-f27b0108d968 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:38.224062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5e06870e-74e4-432f-9c5d-f27b0108d968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:38.265217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 921/1833] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:38.270355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77004b3a-67a7-400c-8653-cc504292bcff None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.283464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77004b3a-67a7-400c-8653-cc504292bcff None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.289807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.292229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.292659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.293106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.293920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:38.294363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.297639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.331447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 913/1834] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:38.336209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06870e-74e4-432f-9c5d-f27b0108d968 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.359112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06870e-74e4-432f-9c5d-f27b0108d968 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.372430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.373809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.376055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.376594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.377603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.379822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.440140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 922/1835] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:38 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:38.441941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.456555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb98a48-6770-44e2-960d-3ba90e79768f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.526198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:38.546863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.548320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.553833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.556869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.557708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.561759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.579125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.580977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:38.624551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9edb03-25b3-4a53-8f71-695b6d5d018f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.628927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 914/1836] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:38 2022] POST /identity/v3/users => generated 281 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:38.633310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.639535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.643955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.644796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.644924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d7a54-694f-4ea8-85dd-3babebb7d0ff admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.645796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] PATH_INFO: `/v3/projects/e00962e815cd4ec2a90135d306bde922` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.648399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00962e815cd4ec2a90135d306bde922)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.649986 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.650604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.651236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.651514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.651787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.652437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.654080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.666961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 915/1837] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:38.671310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.683478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210f708-9586-40f6-9c2e-71c022dee850 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.690072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.690991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.691266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.691627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.692344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.694081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.705659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 916/1838] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:38.710026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.713277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00962e815cd4ec2a90135d306bde922', 'request_id': 'req-678395c6-60c9-4c5d-84a9-6815b8934333'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.723247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad671d59-ac89-4380-b236-0790066702f4 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.727475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00962e815cd4ec2a90135d306bde922', 'request_id': 'req-678395c6-60c9-4c5d-84a9-6815b8934333'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.733872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00962e815cd4ec2a90135d306bde922', 'request_id': 'req-678395c6-60c9-4c5d-84a9-6815b8934333'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.749984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] The token cache is being invalidate because project e00962e815cd4ec2a90135d306bde922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:38.750267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.750955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.752798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 923/1839] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:38 2022] DELETE /identity/v3/projects/e00962e815cd4ec2a90135d306bde922 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:38.757038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:38.758486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:38.793355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:38.808227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361730b-5b7e-48b5-867a-090f600a943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.808992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361730b-5b7e-48b5-867a-090f600a943c admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.809231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361730b-5b7e-48b5-867a-090f600a943c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.809504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361730b-5b7e-48b5-867a-090f600a943c admin admin] PATH_INFO: `/v3/projects/88e2a0e8c6d149aeae9578779632a941/users/63bf9156eb6f4c49af2fb6ae33354f42/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.829574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361730b-5b7e-48b5-867a-090f600a943c admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e2a0e8c6d149aeae9578779632a941, user_id=63bf9156eb6f4c49af2fb6ae33354f42, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.832533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361730b-5b7e-48b5-867a-090f600a943c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.839709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678395c6-60c9-4c5d-84a9-6815b8934333 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:38.846069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:38.847940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.847940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.847940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] PATH_INFO: `/v3/users/cbf175cb56c84d288bcc66b65928cd4a` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.848432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf175cb56c84d288bcc66b65928cd4a)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:38.850473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:38.853481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 917/1840] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:38 2022] PUT /identity/v3/projects/88e2a0e8c6d149aeae9578779632a941/users/63bf9156eb6f4c49af2fb6ae33354f42/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:59:38.863539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:38.863747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:38.864166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:38.910837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] MFA Rules not processed for user `63bf9156eb6f4c49af2fb6ae33354f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:38.912089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:38.917090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf175cb56c84d288bcc66b65928cd4a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.925582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf175cb56c84d288bcc66b65928cd4a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.937502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf175cb56c84d288bcc66b65928cd4a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.944185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf175cb56c84d288bcc66b65928cd4a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.944890 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf175cb56c84d288bcc66b65928cd4a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:38.954275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 924/1841] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:38 2022] DELETE /identity/v3/users/cbf175cb56c84d288bcc66b65928cd4a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:38.985577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 918/1842] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:39.026165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.027958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:39.126338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dca9c-6c19-45b7-b1ce-21f2e38b5edf admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.134031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.134676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.134957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.135226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.263899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.266094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.270153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:39.282452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 925/1843] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:39.298569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.300383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:39.348705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.349968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:39.350159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:39.387774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0978de-c198-4103-9786-47ef96f74997 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.426963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff24de-d840-405a-b552-47c7157acfa9 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.431641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.432320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.432556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.432865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] PATH_INFO: `/v3/projects/9a299ee7d37c40098bfadb5e92ad471f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.433993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a299ee7d37c40098bfadb5e92ad471f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.436304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.451256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.452089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.452415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.452691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] PATH_INFO: `/v3/users/63bf9156eb6f4c49af2fb6ae33354f42` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.454506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63bf9156eb6f4c49af2fb6ae33354f42)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.456459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.487920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a299ee7d37c40098bfadb5e92ad471f', 'request_id': 'req-f6c55dea-553a-4486-8a90-ee8d7420e0bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.495058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a299ee7d37c40098bfadb5e92ad471f', 'request_id': 'req-f6c55dea-553a-4486-8a90-ee8d7420e0bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.500849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a299ee7d37c40098bfadb5e92ad471f', 'request_id': 'req-f6c55dea-553a-4486-8a90-ee8d7420e0bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.507851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63bf9156eb6f4c49af2fb6ae33354f42'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.508490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63bf9156eb6f4c49af2fb6ae33354f42'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.511911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] The token cache is being invalidate because project 9a299ee7d37c40098bfadb5e92ad471f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:39.512226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.512886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6c55dea-553a-4486-8a90-ee8d7420e0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.514871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 926/1844] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:39 2022] DELETE /identity/v3/projects/9a299ee7d37c40098bfadb5e92ad471f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:59:39.517525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63bf9156eb6f4c49af2fb6ae33354f42'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.521577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1ea03-46f6-470b-ba2c-e523a6db8305 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.521854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1ea03-46f6-470b-ba2c-e523a6db8305 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.522124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1ea03-46f6-470b-ba2c-e523a6db8305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.532354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63bf9156eb6f4c49af2fb6ae33354f42'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.539266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d06b639-e946-44fb-8886-95146c74c793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63bf9156eb6f4c49af2fb6ae33354f42'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:39.542696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d8a1ea03-46f6-470b-ba2c-e523a6db8305 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:39.544066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d8a1ea03-46f6-470b-ba2c-e523a6db8305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:39.545520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 919/1845] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:39 2022] DELETE /identity/v3/users/63bf9156eb6f4c49af2fb6ae33354f42 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:39.549803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.550088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.550356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.569449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:39.570645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:39.605701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 927/1846] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:39.613408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66ed10-09c5-4bc2-80b3-92e9740afb47 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.613697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66ed10-09c5-4bc2-80b3-92e9740afb47 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.613972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66ed10-09c5-4bc2-80b3-92e9740afb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.621631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:39.635967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 920/1847] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:39.646767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.648346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-fc66ed10-09c5-4bc2-80b3-92e9740afb47 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:39.649603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fc66ed10-09c5-4bc2-80b3-92e9740afb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:39.652011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:39.659000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 928/1848] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:39.662773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66ed10-09c5-4bc2-80b3-92e9740afb47 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.673693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66ed10-09c5-4bc2-80b3-92e9740afb47 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.678700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.679501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.679935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.680496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.681474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:39.682034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.683985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.712038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 929/1849] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 15:59:39.716945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.720687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cf6b1-a2b6-4795-937a-d6099cb8bbc7 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.725151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.725775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.726050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.726316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.728740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fbe4e-efb0-47bf-8b57-41886ee78f44 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.733659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.734429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.734811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.735245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.735911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.736106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.737593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.738344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.744471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:39.755266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 921/1850] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:39.792246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 930/1851] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:39 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:59:39.796495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.807930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e6843-839d-4705-846b-072318c1be02 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.876852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.877550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.877720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.877983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.878621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.880547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.903377 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 922/1852] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:39 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:39.908004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.922396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41756f9-2c13-4f17-914d-42083024c71e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.928965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.929646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.929988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.930322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.931039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.932761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.946969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 931/1853] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:39.951112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:39.963246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da775c8-52fb-49fb-8f4c-2c5c3e51bcea admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:39.968359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d2dbd-d8aa-4c97-914d-2e994345f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:39.969617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2dbd-d8aa-4c97-914d-2e994345f5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:39.969920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2dbd-d8aa-4c97-914d-2e994345f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:39.970129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2dbd-d8aa-4c97-914d-2e994345f5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:39.970769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d2dbd-d8aa-4c97-914d-2e994345f5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:39.972537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d2dbd-d8aa-4c97-914d-2e994345f5c0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:39.984773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 923/1854] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:39.989043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.002311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4dad2-8be5-48e3-bd49-a4e6415c0238 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.008388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.009494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.009494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.009717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] PATH_INFO: `/v3/projects/312965e990464e75a63877f991456063/users/0f75bffeffa9499faaf0d283f621357b/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.025945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=312965e990464e75a63877f991456063, user_id=0f75bffeffa9499faaf0d283f621357b, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.028652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.047992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 932/1855] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:39 2022] PUT /identity/v3/projects/312965e990464e75a63877f991456063/users/0f75bffeffa9499faaf0d283f621357b/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:40.059126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333c27-01d8-4ee1-8202-22fa8a604795 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.059601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333c27-01d8-4ee1-8202-22fa8a604795 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.060388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29333c27-01d8-4ee1-8202-22fa8a604795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.071864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.074193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:40.116756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-29333c27-01d8-4ee1-8202-22fa8a604795 None None] MFA Rules not processed for user `0f75bffeffa9499faaf0d283f621357b`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:40.117980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-29333c27-01d8-4ee1-8202-22fa8a604795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:40.152371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e43b28-7c6b-4642-8400-7e3cd402af5a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.158289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.158945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.159315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.159664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] PATH_INFO: `/v3/projects/88e2a0e8c6d149aeae9578779632a941` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.160918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e2a0e8c6d149aeae9578779632a941)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.162986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.195334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 924/1856] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:40.199878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29333c27-01d8-4ee1-8202-22fa8a604795 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.212118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29333c27-01d8-4ee1-8202-22fa8a604795 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.222696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:40.227166 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e2a0e8c6d149aeae9578779632a941', 'request_id': 'req-2518fa7d-1650-4f18-aa02-e9327ac5e696'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:40.253634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e2a0e8c6d149aeae9578779632a941', 'request_id': 'req-2518fa7d-1650-4f18-aa02-e9327ac5e696'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:40.263253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e2a0e8c6d149aeae9578779632a941', 'request_id': 'req-2518fa7d-1650-4f18-aa02-e9327ac5e696'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:40.279745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] The token cache is being invalidate because project 88e2a0e8c6d149aeae9578779632a941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:40.280356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:40.282104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2518fa7d-1650-4f18-aa02-e9327ac5e696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:40.284222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 933/1857] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:40 2022] DELETE /identity/v3/projects/88e2a0e8c6d149aeae9578779632a941 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:59:40.289402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578479d-8b97-4cdc-b00e-1360778a7244 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.289763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578479d-8b97-4cdc-b00e-1360778a7244 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.290109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578479d-8b97-4cdc-b00e-1360778a7244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.326443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.327128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.327495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.328592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.329500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.331443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.346721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a578479d-8b97-4cdc-b00e-1360778a7244 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:40.353069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a578479d-8b97-4cdc-b00e-1360778a7244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:40.380782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 925/1858] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:59:40.380782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.380782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:40.381189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 934/1859] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:40.384115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d1cec-cb83-439e-ba71-253b494addca None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.384115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d1cec-cb83-439e-ba71-253b494addca None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.384115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d1cec-cb83-439e-ba71-253b494addca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.404402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-de0d1cec-cb83-439e-ba71-253b494addca None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:40.405936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-de0d1cec-cb83-439e-ba71-253b494addca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:40.449553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:40.465166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768430d6-b555-45fb-adbd-ad8afbb85472 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.471109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.471762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.472042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.472317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.472987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.474941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.495772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 935/1860] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:40.497113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d1cec-cb83-439e-ba71-253b494addca None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.500677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 926/1861] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:40.504654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.508093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d1cec-cb83-439e-ba71-253b494addca None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.512929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.513536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.513816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.514088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.514760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:40.515055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.516225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f073e0-6150-4ec1-a632-96b015733a76 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.516922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.522239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.522669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.523176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.523366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.523952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.525896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.535574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 936/1862] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:40.536479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.538514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 927/1863] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:40.542587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.548066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3afc6-e60c-4689-9580-ec6aad00b10c admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.552980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.553648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.553937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.554219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.554689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44d412-206b-409f-b8d5-0e633a2a7600 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.554868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.557283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.559664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.560336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.560620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.561039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] PATH_INFO: `/v3/projects/d220a411ab824e5aa76a85fcf653205e/users/7ed6b32a1c944cccbdd32b178e61f546/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.580206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d220a411ab824e5aa76a85fcf653205e, user_id=7ed6b32a1c944cccbdd32b178e61f546, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.583272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.597565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 937/1864] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:40.601752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.605771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 928/1865] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:40 2022] PUT /identity/v3/projects/d220a411ab824e5aa76a85fcf653205e/users/7ed6b32a1c944cccbdd32b178e61f546/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:59:40.611452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.673632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c45eb-5c55-4bc1-b992-f34e6326a4c7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.676323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:40.679500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.680454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.680924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.681261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.681967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.684228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.689745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f02daf-e740-4ce3-8137-7299e3a27da3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.696903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.697631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.697983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.698333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.699557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.700805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.725319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 938/1866] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:40.730921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.731332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 929/1867] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:40.738084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.743674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78cc97-b8da-4fbd-81a1-f3ff06c6707a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.750001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.750779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.751162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.751614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.752709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.754709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f70bac-77dd-47c8-871b-5c48386dd23b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.755857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.770058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 939/1868] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:40.773619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e10273-4ff3-44a1-b105-e9976977b985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.774302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10273-4ff3-44a1-b105-e9976977b985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.774415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.774579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10273-4ff3-44a1-b105-e9976977b985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.774882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10273-4ff3-44a1-b105-e9976977b985 admin admin] PATH_INFO: `/v3/projects/d220a411ab824e5aa76a85fcf653205e/users/7ed6b32a1c944cccbdd32b178e61f546/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.776800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e10273-4ff3-44a1-b105-e9976977b985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d220a411ab824e5aa76a85fcf653205e, user_id=7ed6b32a1c944cccbdd32b178e61f546, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.779510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e10273-4ff3-44a1-b105-e9976977b985 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.785807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114b8ce-2e7f-4d1a-9362-4494e2d0ef58 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.791099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 930/1869] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:40 2022] PUT /identity/v3/projects/d220a411ab824e5aa76a85fcf653205e/users/7ed6b32a1c944cccbdd32b178e61f546/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:40.803968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.804429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.804800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.850473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] MFA Rules not processed for user `7ed6b32a1c944cccbdd32b178e61f546`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:40.852141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:40.861981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.862642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.862925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.863197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.863853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.866099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.879472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 940/1870] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:40.880875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.892670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a1e46-d7c8-4874-b9e8-2e12bf4067f4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:40.901518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9fc2c-457a-4c3c-b080-59f5fea70934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:40.902318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9fc2c-457a-4c3c-b080-59f5fea70934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.902603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9fc2c-457a-4c3c-b080-59f5fea70934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.902878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9fc2c-457a-4c3c-b080-59f5fea70934 admin admin] PATH_INFO: `/v3/projects/e45c4d35d6964105aae8049e30f3a64a/users/5b68aeb37ae34d1a83188b6b9ac85b62/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.917483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:40.921701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9fc2c-457a-4c3c-b080-59f5fea70934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45c4d35d6964105aae8049e30f3a64a, user_id=5b68aeb37ae34d1a83188b6b9ac85b62, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:40.925989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9fc2c-457a-4c3c-b080-59f5fea70934 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:40.936096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 931/1871] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:40.943710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 941/1872] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:40 2022] PUT /identity/v3/projects/e45c4d35d6964105aae8049e30f3a64a/users/5b68aeb37ae34d1a83188b6b9ac85b62/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:40.953845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:40.955782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7ec7c-5861-4146-a044-e49edd81568f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:40.956072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7ec7c-5861-4146-a044-e49edd81568f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:40.956332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:40.956922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7ec7c-5861-4146-a044-e49edd81568f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:40.989128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-49b7ec7c-5861-4146-a044-e49edd81568f None None] MFA Rules not processed for user `5b68aeb37ae34d1a83188b6b9ac85b62`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:40.990428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-49b7ec7c-5861-4146-a044-e49edd81568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:41.043950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49a8be-6589-4b96-9801-9d0a1e2b10b2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.077982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 942/1873] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:41.080139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7ec7c-5861-4146-a044-e49edd81568f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.100535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.101491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.101971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.102495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.142443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:41.166189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7ec7c-5861-4146-a044-e49edd81568f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.171410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.172744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.173099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.173459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.174169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.176168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.183920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.185658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.207187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 932/1874] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:41.220481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 943/1875] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:41 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:41.224220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.240916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cbb03-a1c5-4090-af3a-32bc20f75d82 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.377614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.379102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.379393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.379673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.380633 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.383789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.415501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 933/1876] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:41 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:41.419143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.430416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15878dc0-006c-434e-97e9-b106a1fade26 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.441127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.441838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.442308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.442845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.443727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.445579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.460488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 944/1877] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:59:41.462507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.474162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee152d44-97c9-4c7f-9821-3130dc4cd282 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.492276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.493044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.493367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.493649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] PATH_INFO: `/v3/projects/f412c2cc2f0642eab48ab543020b3b3f/users/2833f37d0e9e4d11bff675f8bb73ddde/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.518017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] RBAC: Authorizing `identity:create_grant(project_id=f412c2cc2f0642eab48ab543020b3b3f, user_id=2833f37d0e9e4d11bff675f8bb73ddde, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.523721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.539898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 934/1878] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:41 2022] PUT /identity/v3/projects/f412c2cc2f0642eab48ab543020b3b3f/users/2833f37d0e9e4d11bff675f8bb73ddde/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:41.544041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.555284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c542fe-9c10-4fcd-aafa-2e57604892ed admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.613726 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:41.628555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.630183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.630690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.631302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.632080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.633863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.650373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 945/1879] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:41.651939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.653821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.664552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918c1e7-31ca-4ac8-b4cb-323fb18276da admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.671745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d3a08-630a-402c-8aae-970754def35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.672569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d3a08-630a-402c-8aae-970754def35d admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.672880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d3a08-630a-402c-8aae-970754def35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.673153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d3a08-630a-402c-8aae-970754def35d admin admin] PATH_INFO: `/v3/projects/f412c2cc2f0642eab48ab543020b3b3f/users/2833f37d0e9e4d11bff675f8bb73ddde/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.675136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d3a08-630a-402c-8aae-970754def35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f412c2cc2f0642eab48ab543020b3b3f, user_id=2833f37d0e9e4d11bff675f8bb73ddde, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.678094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d3a08-630a-402c-8aae-970754def35d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.690277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 935/1880] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:41 2022] PUT /identity/v3/projects/f412c2cc2f0642eab48ab543020b3b3f/users/2833f37d0e9e4d11bff675f8bb73ddde/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:41.700383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.703536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.703856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.738202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001bea-4063-4e57-9d04-293d32bc70e0 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.739517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] MFA Rules not processed for user `2833f37d0e9e4d11bff675f8bb73ddde`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:41.741660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:41.842667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 936/1881] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:41.848398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed6b32a1c944cccbdd32b178e61f546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd220a411ab824e5aa76a85fcf653205e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.850296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.850585 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.851217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.852902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:41.853527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.855625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.857121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:41.858200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.864510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:41.880801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 946/1882] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:41.883122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:41.894268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa4dcd-bfc2-421d-9d11-0f2f05b7e306 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.900454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed6b32a1c944cccbdd32b178e61f546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd220a411ab824e5aa76a85fcf653205e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.901253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.901748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.902117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.902958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:41.908218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:41.929224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84f01f-d8a4-4b3b-9300-c55943c97035 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:41.934715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:41.935368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:41.935648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:41.935917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:41.959087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 947/1883] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:41 2022] POST /identity/v3/projects => generated 354 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:59:41.965055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:42.015276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:42.016816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:42.027036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 937/1884] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:42.039879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb78c51-4ef4-4712-b3ba-00db90b0c02f tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:42.080531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed6b32a1c944cccbdd32b178e61f546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd220a411ab824e5aa76a85fcf653205e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:42.081201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:42.081482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:42.081757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:42.082423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:42.082725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:42.084462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:42.112812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 948/1885] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:42.165718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:42.191879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c29d1a-70f5-4ef0-aa3b-f0b97de31ad2 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:42.203710 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed6b32a1c944cccbdd32b178e61f546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd220a411ab824e5aa76a85fcf653205e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:42.204507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:42.204893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:42.205166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] PATH_INFO: `/v3/projects/78ea27cbf19b48b5820d10636887f2d0` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:42.207495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorizing `identity:delete_project(project_id=78ea27cbf19b48b5820d10636887f2d0)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:42.209852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:42.291122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ea27cbf19b48b5820d10636887f2d0', 'request_id': 'req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:42.302189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ea27cbf19b48b5820d10636887f2d0', 'request_id': 'req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:42.324186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ea27cbf19b48b5820d10636887f2d0', 'request_id': 'req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:42.341158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] The token cache is being invalidate because project 78ea27cbf19b48b5820d10636887f2d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:42.341456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:42.342307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:42.344645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 938/1886] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:42 2022] DELETE /identity/v3/projects/78ea27cbf19b48b5820d10636887f2d0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:42.440007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:42.444858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:42.457086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:42.614731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5a05f-081e-4ff9-8130-40d1bf8fb086 tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:42.771571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b80d02-1ef5-4971-a2c9-fa48cb6b9a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:42.772310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80d02-1ef5-4971-a2c9-fa48cb6b9a33 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:42.772631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80d02-1ef5-4971-a2c9-fa48cb6b9a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:42.772956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80d02-1ef5-4971-a2c9-fa48cb6b9a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:42.775400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-32b80d02-1ef5-4971-a2c9-fa48cb6b9a33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:42.988384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b80d02-1ef5-4971-a2c9-fa48cb6b9a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:42.996462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b80d02-1ef5-4971-a2c9-fa48cb6b9a33 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:43.005062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:43.007730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:43.018944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 949/1887] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:43.137857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:43.163329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90c5cb-dfa3-482e-96c0-ff519b7fe61c tempest-VolumeQuotaClassesTest-1962528531 tempest-VolumeQuotaClassesTest-1962528531-project-admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:43.167887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:43.168514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:43.168819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:43.169091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] PATH_INFO: `/v3/users/0f75bffeffa9499faaf0d283f621357b` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:43.170225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f75bffeffa9499faaf0d283f621357b)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:43.172565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:43.242848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f75bffeffa9499faaf0d283f621357b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:43.243848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f75bffeffa9499faaf0d283f621357b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:43.253510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f75bffeffa9499faaf0d283f621357b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:43.269709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f75bffeffa9499faaf0d283f621357b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:43.277752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-704ade92-0995-4065-bbfe-5e70e2e6fd48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f75bffeffa9499faaf0d283f621357b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:43.289821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 939/1888] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:43 2022] DELETE /identity/v3/users/0f75bffeffa9499faaf0d283f621357b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:43.294463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008666-033b-495b-be58-8f519ca98db4 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:43.294912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008666-033b-495b-be58-8f519ca98db4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:43.295477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008666-033b-495b-be58-8f519ca98db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:43.329140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-38008666-033b-495b-be58-8f519ca98db4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:43.330515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-38008666-033b-495b-be58-8f519ca98db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:43.424865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f23b-b57f-465e-bad6-274ea022e896 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:43.426078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f23b-b57f-465e-bad6-274ea022e896 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:43.426371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f23b-b57f-465e-bad6-274ea022e896 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:43.428359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 940/1889] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 15:59:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 15:59:43.429960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 950/1890] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:43.441604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:43.441857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:43.442213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:43.484660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:43.490302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:43.563725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008666-033b-495b-be58-8f519ca98db4 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:43.565381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-38008666-033b-495b-be58-8f519ca98db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:43.591922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:43.687168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008666-033b-495b-be58-8f519ca98db4 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:43.691969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 941/1891] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:43.696798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:43.697714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:43.698098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:43.698474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:43.711143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:43.713014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:43.725786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 951/1892] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 15:59:43.755749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:43.757232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:43.782137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbdd39-9c93-4508-90d0-c70cc824418d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:43.800869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:43.801419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:43.801989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:43.802284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:43.813867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:43.815889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:43.841517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 942/1893] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:44.382265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:44.383749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:44.397515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff13de-19a2-4693-9f30-ff3f61f489fa service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:44.402188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:44.402815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:44.403103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:44.403379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] PATH_INFO: `/v3/projects/312965e990464e75a63877f991456063` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:44.404552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=312965e990464e75a63877f991456063)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:44.406856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:44.461467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312965e990464e75a63877f991456063', 'request_id': 'req-7ef54e43-b1e8-472f-8582-0468a7687d37'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.470676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312965e990464e75a63877f991456063', 'request_id': 'req-7ef54e43-b1e8-472f-8582-0468a7687d37'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.477357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312965e990464e75a63877f991456063', 'request_id': 'req-7ef54e43-b1e8-472f-8582-0468a7687d37'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.490419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] The token cache is being invalidate because project 312965e990464e75a63877f991456063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:44.490932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.491692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.494083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 952/1894] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:44 2022] DELETE /identity/v3/projects/312965e990464e75a63877f991456063 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:44.498706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:44.511010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49e7be-dc0b-41eb-8fa7-16c35e9e085f service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:44.549485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:44.582174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:44.582811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:44.583082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:44.583342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] PATH_INFO: `/v3/users/7ed6b32a1c944cccbdd32b178e61f546` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:44.584468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed6b32a1c944cccbdd32b178e61f546)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:44.586417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:44.644193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed6b32a1c944cccbdd32b178e61f546'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.644931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed6b32a1c944cccbdd32b178e61f546'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.657884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed6b32a1c944cccbdd32b178e61f546'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.671258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed6b32a1c944cccbdd32b178e61f546'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.681690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ba9fdab0-c241-430e-a39b-c57e0b0ba938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed6b32a1c944cccbdd32b178e61f546'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:44.689182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 943/1895] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:44 2022] DELETE /identity/v3/users/7ed6b32a1c944cccbdd32b178e61f546 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:44.980944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:44.982653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:45.005380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54e43-b1e8-472f-8582-0468a7687d37 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.079518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.080190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.080448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.080738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] PATH_INFO: `/v3/projects/d220a411ab824e5aa76a85fcf653205e` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.081967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d220a411ab824e5aa76a85fcf653205e)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.083927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.179163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:45.195767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd220a411ab824e5aa76a85fcf653205e', 'request_id': 'req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:45.212620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd220a411ab824e5aa76a85fcf653205e', 'request_id': 'req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:45.220909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd220a411ab824e5aa76a85fcf653205e', 'request_id': 'req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:45.235997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] The token cache is being invalidate because project d220a411ab824e5aa76a85fcf653205e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:45.236290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:45.237123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-4c38875c-ee8d-42fe-94ee-fcd69e26bd4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:45.238953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 953/1896] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:44 2022] DELETE /identity/v3/projects/d220a411ab824e5aa76a85fcf653205e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:45.244042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880092-aae6-4071-83f2-db23d6856b0d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.244325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880092-aae6-4071-83f2-db23d6856b0d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.244602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1880092-aae6-4071-83f2-db23d6856b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.284492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-f1880092-aae6-4071-83f2-db23d6856b0d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:45.285764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f1880092-aae6-4071-83f2-db23d6856b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:45.364883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 944/1897] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:45.369836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411aa6-1291-46ac-b813-51673a4bb0af None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.370116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411aa6-1291-46ac-b813-51673a4bb0af None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.370393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51411aa6-1291-46ac-b813-51673a4bb0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.392977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-51411aa6-1291-46ac-b813-51673a4bb0af None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:45.393998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-51411aa6-1291-46ac-b813-51673a4bb0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:45.402108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 954/1898] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:45.407080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880092-aae6-4071-83f2-db23d6856b0d None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:45.417371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1880092-aae6-4071-83f2-db23d6856b0d None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.422422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.423021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.423295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.423562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.424206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:45.424504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.426180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.439416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 945/1899] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:45.443741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51411aa6-1291-46ac-b813-51673a4bb0af None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:45.453759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51411aa6-1291-46ac-b813-51673a4bb0af None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.458507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.459157 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.459439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.459709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.460354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.462344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.494571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 955/1900] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:45 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:45.496028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:45.506613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8d10d-9603-493a-b4cf-8759cdcb4606 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.548344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:45.566450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.567490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.567769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.568045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.568703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.570681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.595667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 946/1901] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:45 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:45.614089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:45.623390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216515-002e-4308-81d7-09d7e49b6d56 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.628745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.629368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.629640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.629907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.630550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.632150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.643842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 956/1902] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:45.647897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:45.659823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab1537-5077-4827-9581-adbea1b329b1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.664537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023171eb-f11f-4267-bb48-7879845d3cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.665215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023171eb-f11f-4267-bb48-7879845d3cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.665497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023171eb-f11f-4267-bb48-7879845d3cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.665771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023171eb-f11f-4267-bb48-7879845d3cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.666425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023171eb-f11f-4267-bb48-7879845d3cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.668073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023171eb-f11f-4267-bb48-7879845d3cf5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.683579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 947/1903] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:45.684825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:45.696476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806823c-d1bb-467f-a0c4-ec490de338e0 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.701460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.702167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.702474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.703430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] PATH_INFO: `/v3/projects/33908f77b1754ea6b63aabef54f89e16/users/102b8d3e209f4914b1aff11392384e24/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.721390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33908f77b1754ea6b63aabef54f89e16, user_id=102b8d3e209f4914b1aff11392384e24, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.724097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.742848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 957/1904] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:45 2022] PUT /identity/v3/projects/33908f77b1754ea6b63aabef54f89e16/users/102b8d3e209f4914b1aff11392384e24/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:45.755655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.755943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.756246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.787299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] MFA Rules not processed for user `102b8d3e209f4914b1aff11392384e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:45.788526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:45.857735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 948/1905] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:45.870049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:45.871826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:45.941887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06b391-9161-4c9e-8ec1-1c4b1e48f277 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:45.946134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16f728-5e0f-4124-84d6-953afe377726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:45.946751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f728-5e0f-4124-84d6-953afe377726 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:45.947035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f728-5e0f-4124-84d6-953afe377726 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:45.947305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16f728-5e0f-4124-84d6-953afe377726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:45.957259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16f728-5e0f-4124-84d6-953afe377726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:45.958705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16f728-5e0f-4124-84d6-953afe377726 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:45.979365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 958/1906] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:47.284859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:47.289967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:47.395524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859185a-f6f0-4ad7-8321-440425bc6d83 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:47.401474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:47.403084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:47.403788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:47.403788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] PATH_INFO: `/v3/users/5b68aeb37ae34d1a83188b6b9ac85b62` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:47.405399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b68aeb37ae34d1a83188b6b9ac85b62)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:47.409369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:47.509025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b68aeb37ae34d1a83188b6b9ac85b62'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:47.510144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b68aeb37ae34d1a83188b6b9ac85b62'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:47.522925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b68aeb37ae34d1a83188b6b9ac85b62'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:47.525099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:47.568567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b68aeb37ae34d1a83188b6b9ac85b62'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:47.596241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b68aeb37ae34d1a83188b6b9ac85b62'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:47.618911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 949/1907] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:47 2022] DELETE /identity/v3/users/5b68aeb37ae34d1a83188b6b9ac85b62 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:47.623787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:47.623787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:47.624362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:47.680688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:47.682028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:47.856006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 959/1908] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:47.905039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:47.907629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:48.116841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7add46-bfbb-4187-ab8c-c0251df0d7d8 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:48.136988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:48.137641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:48.137924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:48.138197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:48.156432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:48.158172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:48.185459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 950/1909] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:49.041674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:49.044414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:49.063095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaac78c-17c0-4df3-84a7-4849bcd81b31 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:49.069695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:49.070573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:49.071041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:49.071499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] PATH_INFO: `/v3/projects/e45c4d35d6964105aae8049e30f3a64a` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:49.072936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45c4d35d6964105aae8049e30f3a64a)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:49.075140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:49.076880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:49.150561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45c4d35d6964105aae8049e30f3a64a', 'request_id': 'req-3890b1aa-6f58-4e9f-8551-10cb610a5dea'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.163124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45c4d35d6964105aae8049e30f3a64a', 'request_id': 'req-3890b1aa-6f58-4e9f-8551-10cb610a5dea'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.172044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45c4d35d6964105aae8049e30f3a64a', 'request_id': 'req-3890b1aa-6f58-4e9f-8551-10cb610a5dea'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.188561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] The token cache is being invalidate because project e45c4d35d6964105aae8049e30f3a64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:49.188891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.191781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.201905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 960/1910] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:49 2022] DELETE /identity/v3/projects/e45c4d35d6964105aae8049e30f3a64a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:49.204041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:49.219065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ebb5-c5f3-42f6-a230-48b4a258655a service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:49.349755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:49.350509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:49.350857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:49.351202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] PATH_INFO: `/v3/users/2833f37d0e9e4d11bff675f8bb73ddde` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:49.353312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2833f37d0e9e4d11bff675f8bb73ddde)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:49.357593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:49.427211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2833f37d0e9e4d11bff675f8bb73ddde'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.428154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2833f37d0e9e4d11bff675f8bb73ddde'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.440152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2833f37d0e9e4d11bff675f8bb73ddde'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.465385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2833f37d0e9e4d11bff675f8bb73ddde'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.493557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:49.498660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-0b74787e-72e5-4dff-9c04-f189fe76d6f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2833f37d0e9e4d11bff675f8bb73ddde'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:49.508327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 951/1911] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 15:59:49 2022] DELETE /identity/v3/users/2833f37d0e9e4d11bff675f8bb73ddde => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:49.893875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:49.895551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:49.920130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890b1aa-6f58-4e9f-8551-10cb610a5dea admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.027683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.028471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.028852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.029196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] PATH_INFO: `/v3/projects/f412c2cc2f0642eab48ab543020b3b3f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.030427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f412c2cc2f0642eab48ab543020b3b3f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.033472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.097439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f412c2cc2f0642eab48ab543020b3b3f', 'request_id': 'req-34a11f26-9c31-40a0-bc66-e7fe2d79be54'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:50.104769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f412c2cc2f0642eab48ab543020b3b3f', 'request_id': 'req-34a11f26-9c31-40a0-bc66-e7fe2d79be54'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:50.110748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f412c2cc2f0642eab48ab543020b3b3f', 'request_id': 'req-34a11f26-9c31-40a0-bc66-e7fe2d79be54'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:50.122919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] The token cache is being invalidate because project f412c2cc2f0642eab48ab543020b3b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 15:59:50.123205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:50.123972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-34a11f26-9c31-40a0-bc66-e7fe2d79be54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 15:59:50.126183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 961/1912] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 15:59:49 2022] DELETE /identity/v3/projects/f412c2cc2f0642eab48ab543020b3b3f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:50.131339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2437-ad8d-40e0-93db-7bb377c52db1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.131775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2437-ad8d-40e0-93db-7bb377c52db1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.131998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2437-ad8d-40e0-93db-7bb377c52db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.152438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-530d2437-ad8d-40e0-93db-7bb377c52db1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:50.153754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-530d2437-ad8d-40e0-93db-7bb377c52db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:50.212901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 952/1913] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:50.220582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836172-dd86-4067-a793-5dba2652c5c8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.220905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836172-dd86-4067-a793-5dba2652c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.221175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43836172-dd86-4067-a793-5dba2652c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.240700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-43836172-dd86-4067-a793-5dba2652c5c8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:50.241922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-43836172-dd86-4067-a793-5dba2652c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:50.250755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 962/1914] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:50.254690 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2437-ad8d-40e0-93db-7bb377c52db1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.266416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2437-ad8d-40e0-93db-7bb377c52db1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.271224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.271862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.272138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.272511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.273243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 15:59:50.273545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.275145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.286885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 953/1915] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 15:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 15:59:50.291263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836172-dd86-4067-a793-5dba2652c5c8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.304617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43836172-dd86-4067-a793-5dba2652c5c8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.309340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.309979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.310257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.310620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.311295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.313973 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.330913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:50.353948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 963/1916] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:50 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:50.355423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.365903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34c325-68cd-40ae-9a2d-576e46086372 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.423802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.424866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.425322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.425852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.426715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.429012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.454470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 954/1917] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:50 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:50.458400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.470440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453be28-bf72-40a3-a062-86633e813151 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.475338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.475940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.476220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.476491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.477239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.478833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.493000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 964/1918] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:50.494036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.504868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769259fb-856b-4113-b1dc-3e7f4a940a9e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.509812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cc5ec-01ba-463f-a802-916113e3014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.510409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cc5ec-01ba-463f-a802-916113e3014b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.510682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cc5ec-01ba-463f-a802-916113e3014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.510950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cc5ec-01ba-463f-a802-916113e3014b admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.511593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cc5ec-01ba-463f-a802-916113e3014b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.513250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cc5ec-01ba-463f-a802-916113e3014b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.524946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 955/1919] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:50.529428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.540317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8feea5-d178-425c-b0f2-d5b2b074f72a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.544956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.545603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.545879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.546194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] PATH_INFO: `/v3/projects/8e8ce25a208e48a6bbf2a855e8f07336/users/d89c736ca1f241efb194ff9031f766b0/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.562659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8ce25a208e48a6bbf2a855e8f07336, user_id=d89c736ca1f241efb194ff9031f766b0, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.565859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.579663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 965/1920] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:50 2022] PUT /identity/v3/projects/8e8ce25a208e48a6bbf2a855e8f07336/users/d89c736ca1f241efb194ff9031f766b0/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:50.589931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fbe3f-9854-442c-a8c9-a14da38e5a86 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.590213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fbe3f-9854-442c-a8c9-a14da38e5a86 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.590479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fbe3f-9854-442c-a8c9-a14da38e5a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.618591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-596fbe3f-9854-442c-a8c9-a14da38e5a86 None None] MFA Rules not processed for user `d89c736ca1f241efb194ff9031f766b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:50.619807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-596fbe3f-9854-442c-a8c9-a14da38e5a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:50.668682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:50.691757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 956/1921] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:50.696483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.767116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e3cb3-c7a7-46fa-85a0-028c6cc56b63 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.771914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.772523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.772864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.773144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.773780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.775751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.805449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 966/1922] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:50 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:59:50.809559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fbe3f-9854-442c-a8c9-a14da38e5a86 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.821308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fbe3f-9854-442c-a8c9-a14da38e5a86 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.904959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.904959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.904959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.904959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.905331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.905331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.929805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 957/1923] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:50 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:50.930982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.941804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab473f2-947c-4b42-99ca-b705b649897f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.946351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.946936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.947206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.947470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.948102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.949727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.961520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 967/1924] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:50.965705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:50.978645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23615322-1725-40c5-bb66-1e9e0daec15f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:50.983402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e8488-c3ba-43a1-9a5e-aa9a7bbb8769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:50.984144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8488-c3ba-43a1-9a5e-aa9a7bbb8769 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:50.984624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8488-c3ba-43a1-9a5e-aa9a7bbb8769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:50.985021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8488-c3ba-43a1-9a5e-aa9a7bbb8769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:50.985658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e8488-c3ba-43a1-9a5e-aa9a7bbb8769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:50.987271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e8488-c3ba-43a1-9a5e-aa9a7bbb8769 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:50.998084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 958/1925] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 15:59:51.004256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.017253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71750d2-a641-4f1d-89ee-aa0ad4122829 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.021926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.022576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.022849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.023116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] PATH_INFO: `/v3/projects/b3ae0dfa20da4f73abee8c4b000ec207/users/136a55dbe4be40439d0d607bf61f08bb/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.038304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ae0dfa20da4f73abee8c4b000ec207, user_id=136a55dbe4be40439d0d607bf61f08bb, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.040982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.054768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 968/1926] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:51 2022] PUT /identity/v3/projects/b3ae0dfa20da4f73abee8c4b000ec207/users/136a55dbe4be40439d0d607bf61f08bb/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:51.067069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7103ffe-2a0c-4ca3-bf1d-13427a07fae3 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.067352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7103ffe-2a0c-4ca3-bf1d-13427a07fae3 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.067623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7103ffe-2a0c-4ca3-bf1d-13427a07fae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.100376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c7103ffe-2a0c-4ca3-bf1d-13427a07fae3 None None] MFA Rules not processed for user `136a55dbe4be40439d0d607bf61f08bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:51.101735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c7103ffe-2a0c-4ca3-bf1d-13427a07fae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:51.185811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 959/1927] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 15:59:51.188628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.254210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf615d1-0558-4210-a04f-661656fcea88 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.259039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.259642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.259919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.260189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.260862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.262763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.274686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:51.292508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 969/1928] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 15:59:51 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 15:59:51.297207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7103ffe-2a0c-4ca3-bf1d-13427a07fae3 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.308446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7103ffe-2a0c-4ca3-bf1d-13427a07fae3 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.366062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:51.373341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.373948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.374236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.375448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.375823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.377796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.401860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 960/1929] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 15:59:51 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 15:59:51.403096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.413126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710ecf-d532-42e7-baaf-2fd9eae2c56f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.417584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.418169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.418444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.418706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.419334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.420955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.431018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 970/1930] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:51.435033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.446038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0581ce8-640d-423b-98f4-c2f27f1cc9a1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.450778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.451422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.451695 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.451983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] PATH_INFO: `/v3/projects/353849cf7a6e4df694699d8a1e884e75/users/4e660dda13f044aa8f8d680be03cdd7d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.466711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=353849cf7a6e4df694699d8a1e884e75, user_id=4e660dda13f044aa8f8d680be03cdd7d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.470075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.482658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 961/1931] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:51 2022] PUT /identity/v3/projects/353849cf7a6e4df694699d8a1e884e75/users/4e660dda13f044aa8f8d680be03cdd7d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 15:59:51.487259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.497081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c664a-ae6a-4f60-8748-d2b7a6b0c031 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.554587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3607092-7f51-4587-b952-6b387577d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.555236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607092-7f51-4587-b952-6b387577d9ea admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.555515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607092-7f51-4587-b952-6b387577d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.555783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3607092-7f51-4587-b952-6b387577d9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.556421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3607092-7f51-4587-b952-6b387577d9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.558037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3607092-7f51-4587-b952-6b387577d9ea admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.568291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 971/1932] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 15:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 15:59:51.572622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.582857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cf7fd-f9a8-42d7-9453-f8a7933203ce admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.593923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.594602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.594884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.595158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] PATH_INFO: `/v3/projects/353849cf7a6e4df694699d8a1e884e75/users/4e660dda13f044aa8f8d680be03cdd7d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.597321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=353849cf7a6e4df694699d8a1e884e75, user_id=4e660dda13f044aa8f8d680be03cdd7d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.600189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.611782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 962/1933] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 15:59:51 2022] PUT /identity/v3/projects/353849cf7a6e4df694699d8a1e884e75/users/4e660dda13f044aa8f8d680be03cdd7d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 15:59:51.622176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.622467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.622742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] MFA Rules not processed for user `4e660dda13f044aa8f8d680be03cdd7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 15:59:51.652389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:51.719178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 972/1934] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 15:59:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 15:59:51.730555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:51.732224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:51.791330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef21f-9230-4efc-b3ba-0d2bac98ab40 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:51.794965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:51.795572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:51.795850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:51.796385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:51.851925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:51.853331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:51.862648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 963/1935] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:53.599188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:53.600820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 15:59:53.631289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005c6f4-c482-4f16-9f77-57b33a8b3d3b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:53.635102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:53.635830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:53.636206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:53.636572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:53.736177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:53.743622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:53.749788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 15:59:53.760077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 973/1936] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 15:59:54.570309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 15:59:54.579863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7746144-7b70-4b2f-870a-344ca10783ae service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 15:59:54.585268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983195a-2e2f-40d4-ace2-097b15046891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 15:59:54.586225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983195a-2e2f-40d4-ace2-097b15046891 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 15:59:54.586510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983195a-2e2f-40d4-ace2-097b15046891 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 15:59:54.586780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983195a-2e2f-40d4-ace2-097b15046891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 15:59:54.599853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983195a-2e2f-40d4-ace2-097b15046891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 15:59:54.601536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983195a-2e2f-40d4-ace2-097b15046891 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 15:59:54.614142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 964/1937] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 15:59:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:00.954254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:01.041415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62667a03-9c74-4b97-a78f-5e49ffec6e63 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:01.046610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:01.047538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:01.047942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:01.048286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] PATH_INFO: `/v3/users/d89c736ca1f241efb194ff9031f766b0` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:01.049458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89c736ca1f241efb194ff9031f766b0)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:01.051772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:01.109159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89c736ca1f241efb194ff9031f766b0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.118011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89c736ca1f241efb194ff9031f766b0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.132236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89c736ca1f241efb194ff9031f766b0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.141906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89c736ca1f241efb194ff9031f766b0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.142812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89c736ca1f241efb194ff9031f766b0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.149260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 974/1938] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:00 2022] DELETE /identity/v3/users/d89c736ca1f241efb194ff9031f766b0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:01.153925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:01.154369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:01.154468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:01.175144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:01.176376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:01.233191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:01.255640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 965/1939] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:01.269495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:01.274143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:01.376847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20184cda-b592-46aa-83aa-c2c13783da63 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:01.380593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:01.381421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:01.381708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:01.381982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:01.396302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:01.397993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:01.420600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 975/1940] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:01.743569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:01.745703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:01.776059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76a844-425f-43f3-8dfd-b0523a765631 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:01.782678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:01.783421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:01.783696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:01.784246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] PATH_INFO: `/v3/projects/8e8ce25a208e48a6bbf2a855e8f07336` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:01.785491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8ce25a208e48a6bbf2a855e8f07336)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:01.787623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:01.839804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8ce25a208e48a6bbf2a855e8f07336', 'request_id': 'req-10226a84-3f4a-4d6a-ae39-85192a83d0e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.847212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8ce25a208e48a6bbf2a855e8f07336', 'request_id': 'req-10226a84-3f4a-4d6a-ae39-85192a83d0e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.853521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8ce25a208e48a6bbf2a855e8f07336', 'request_id': 'req-10226a84-3f4a-4d6a-ae39-85192a83d0e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.866288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] The token cache is being invalidate because project 8e8ce25a208e48a6bbf2a855e8f07336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:01.866579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.867290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:01.868906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 966/1941] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:01 2022] DELETE /identity/v3/projects/8e8ce25a208e48a6bbf2a855e8f07336 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:01.873086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:01.883445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5194ac7e-dd79-44d5-96ac-17680fabcead service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:01.901665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:01.944475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:01.945264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:01.945562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:01.945811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] PATH_INFO: `/v3/users/136a55dbe4be40439d0d607bf61f08bb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:01.947322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=136a55dbe4be40439d0d607bf61f08bb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:01.949328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:02.004751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136a55dbe4be40439d0d607bf61f08bb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.013227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136a55dbe4be40439d0d607bf61f08bb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.026583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136a55dbe4be40439d0d607bf61f08bb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.035356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136a55dbe4be40439d0d607bf61f08bb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.036077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136a55dbe4be40439d0d607bf61f08bb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.042185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 976/1942] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:01 2022] DELETE /identity/v3/users/136a55dbe4be40439d0d607bf61f08bb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:02.152317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:02.154006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:02.167126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10226a84-3f4a-4d6a-ae39-85192a83d0e7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:02.232753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:02.233443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:02.233710 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:02.233983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] PATH_INFO: `/v3/users/102b8d3e209f4914b1aff11392384e24` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:02.235129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=102b8d3e209f4914b1aff11392384e24)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:02.237138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:02.288865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '102b8d3e209f4914b1aff11392384e24'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.289737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '102b8d3e209f4914b1aff11392384e24'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.310174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '102b8d3e209f4914b1aff11392384e24'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.356073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:02.361574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:02.364891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '102b8d3e209f4914b1aff11392384e24'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.369888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:02.374407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-058292bb-4726-427b-a23e-7fd6342a8955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '102b8d3e209f4914b1aff11392384e24'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.381816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea282-0cd8-4c5a-abc9-fe975e02fe05 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:02.387909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 967/1943] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:02 2022] DELETE /identity/v3/users/102b8d3e209f4914b1aff11392384e24 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:02.393321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:02.393615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:02.393988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:02.398626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:02.399266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:02.399554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:02.399817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] PATH_INFO: `/v3/projects/b3ae0dfa20da4f73abee8c4b000ec207` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:02.401051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ae0dfa20da4f73abee8c4b000ec207)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:02.403008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:02.416358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:02.417596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:02.460351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ae0dfa20da4f73abee8c4b000ec207', 'request_id': 'req-96206220-b53a-4e80-aadd-e49728e8d48e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.469529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ae0dfa20da4f73abee8c4b000ec207', 'request_id': 'req-96206220-b53a-4e80-aadd-e49728e8d48e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.476530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ae0dfa20da4f73abee8c4b000ec207', 'request_id': 'req-96206220-b53a-4e80-aadd-e49728e8d48e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.488367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 968/1944] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:02.489216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] The token cache is being invalidate because project b3ae0dfa20da4f73abee8c4b000ec207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:02.489625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.490453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.496232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 977/1945] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:02 2022] DELETE /identity/v3/projects/b3ae0dfa20da4f73abee8c4b000ec207 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:02.497237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:02.498903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:02.499570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:02.501234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:02.512467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eeb475-4a2b-418a-a5fb-4301b1ab0997 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:02.533251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:02.574780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206220-b53a-4e80-aadd-e49728e8d48e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:02.578556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64034723-5459-484a-9052-347e715557d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:02.579197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64034723-5459-484a-9052-347e715557d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:02.579472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64034723-5459-484a-9052-347e715557d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:02.579741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64034723-5459-484a-9052-347e715557d3 admin admin] PATH_INFO: `/v3/users/4e660dda13f044aa8f8d680be03cdd7d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:02.580095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:02.580944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64034723-5459-484a-9052-347e715557d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e660dda13f044aa8f8d680be03cdd7d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:02.581116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:02.581531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:02.581976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:02.583045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64034723-5459-484a-9052-347e715557d3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:02.585608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:02.598933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:02.600545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:02.612529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 978/1946] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:02.638619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e660dda13f044aa8f8d680be03cdd7d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.639852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e660dda13f044aa8f8d680be03cdd7d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.650402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e660dda13f044aa8f8d680be03cdd7d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.664155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e660dda13f044aa8f8d680be03cdd7d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.674465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e660dda13f044aa8f8d680be03cdd7d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:02.688767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 969/1947] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:02 2022] DELETE /identity/v3/users/4e660dda13f044aa8f8d680be03cdd7d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:03.026704 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.028479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.045341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af1af0-1afa-48ab-bdc0-06cf9d7666e4 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.087600 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.089311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.110804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:03.146035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.146985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.147266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.147546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] PATH_INFO: `/v3/projects/33908f77b1754ea6b63aabef54f89e16` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.148916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33908f77b1754ea6b63aabef54f89e16)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.151454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.198201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64034723-5459-484a-9052-347e715557d3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.211263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33908f77b1754ea6b63aabef54f89e16', 'request_id': 'req-c9c3857d-10ba-40e9-b271-7e7913194f32'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.220995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33908f77b1754ea6b63aabef54f89e16', 'request_id': 'req-c9c3857d-10ba-40e9-b271-7e7913194f32'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.228880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33908f77b1754ea6b63aabef54f89e16', 'request_id': 'req-c9c3857d-10ba-40e9-b271-7e7913194f32'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.242347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] The token cache is being invalidate because project 33908f77b1754ea6b63aabef54f89e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:03.242738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.243456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c9c3857d-10ba-40e9-b271-7e7913194f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.245276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 979/1948] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:03 2022] DELETE /identity/v3/projects/33908f77b1754ea6b63aabef54f89e16 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:03.251542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304dd8-3417-4665-b995-8ed05843853b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.251820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304dd8-3417-4665-b995-8ed05843853b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.252709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304dd8-3417-4665-b995-8ed05843853b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.272783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.273561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.274013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.274439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] PATH_INFO: `/v3/projects/353849cf7a6e4df694699d8a1e884e75` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.274601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-59304dd8-3417-4665-b995-8ed05843853b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:03.284752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] RBAC: Authorizing `identity:delete_project(project_id=353849cf7a6e4df694699d8a1e884e75)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.284930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-59304dd8-3417-4665-b995-8ed05843853b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.285051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.286023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 980/1949] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:03.293668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a55f3-cf9d-41a9-969d-186a4d0b3215 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.293957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a55f3-cf9d-41a9-969d-186a4d0b3215 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.294228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a55f3-cf9d-41a9-969d-186a4d0b3215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.316467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-254a55f3-cf9d-41a9-969d-186a4d0b3215 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:03.317982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-254a55f3-cf9d-41a9-969d-186a4d0b3215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.331895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353849cf7a6e4df694699d8a1e884e75', 'request_id': 'req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.340885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353849cf7a6e4df694699d8a1e884e75', 'request_id': 'req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.347044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353849cf7a6e4df694699d8a1e884e75', 'request_id': 'req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.360753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] The token cache is being invalidate because project 353849cf7a6e4df694699d8a1e884e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:03.361131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.361792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a30b96f0-b4e3-4741-9cfe-4f16bbea183b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:03.363311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 970/1950] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:03 2022] DELETE /identity/v3/projects/353849cf7a6e4df694699d8a1e884e75 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:03.369944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99871ba-c72b-4144-bc58-10287219c2ee None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.370304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99871ba-c72b-4144-bc58-10287219c2ee None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.370587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99871ba-c72b-4144-bc58-10287219c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.370733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:03.400952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 981/1951] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:03.405169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a55f3-cf9d-41a9-969d-186a4d0b3215 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.406515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c99871ba-c72b-4144-bc58-10287219c2ee None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:03.408920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c99871ba-c72b-4144-bc58-10287219c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.414875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a55f3-cf9d-41a9-969d-186a4d0b3215 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.419765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.420372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.421293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.421746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.422408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:03.422711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.424305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.425924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 971/1952] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:03.433877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef901-6180-4f59-a8bb-f141a1db6ddb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.434293 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef901-6180-4f59-a8bb-f141a1db6ddb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.434662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef901-6180-4f59-a8bb-f141a1db6ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.438839 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 982/1953] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:03.444786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.453981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75eba0-d1f6-4419-87d0-383406f73d80 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.461522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.461522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.461522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.461522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.461873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.462546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.463726 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a47ef901-6180-4f59-a8bb-f141a1db6ddb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:03.465035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a47ef901-6180-4f59-a8bb-f141a1db6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.474031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 972/1954] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:03.478762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ef901-6180-4f59-a8bb-f141a1db6ddb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.488879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ef901-6180-4f59-a8bb-f141a1db6ddb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.492457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 983/1955] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:03.494438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.495045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.495320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.495587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.496233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:03.496531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.496625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.498296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.513804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 973/1956] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:00:03.515437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.526395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1ea48-9c68-4fba-ba7c-33fec1c9478f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.547601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:03.560114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82296c-03d8-439a-8962-5f6dd4e9211e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.566337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.566944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.567219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.567485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.568139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.570156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.590041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.590673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.590954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.591221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.591846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.594071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.594635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 984/1957] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:03.598698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.610107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28e308-3265-4a38-94d5-09c9cccbadd6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.615030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.615787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.616065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.616338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.617015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.618613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.626496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 974/1958] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:03.627497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.632656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 985/1959] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:03.633625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.643696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84514ea2-1f64-428f-bb92-9a4eb8aa11d6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.697062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086bad9c-f2f2-4615-b74c-bc54824932d0 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.701721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.703088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.703263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.703414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.703646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.703872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.704151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.704272 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.704415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.705112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.706227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.706661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.723842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 986/1960] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:03.725567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.735197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 975/1961] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:03.735530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.738859 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9b400-5a23-4ded-b211-f0d680178c59 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.743601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e78e-8bb6-439e-9ea7-5e9d6c544627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.744261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e78e-8bb6-439e-9ea7-5e9d6c544627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.744538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e78e-8bb6-439e-9ea7-5e9d6c544627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.744835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e78e-8bb6-439e-9ea7-5e9d6c544627 admin admin] PATH_INFO: `/v3/projects/60b016d4443142d4bb81dae536748cf1/users/950e06d7bfff4259972ae1312e8f913a/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.747294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75eee40-31f1-4b7f-baa6-7dfadd132ee3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.753814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.754566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.754942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.755840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.756598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.758356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.773107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8e78e-8bb6-439e-9ea7-5e9d6c544627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b016d4443142d4bb81dae536748cf1, user_id=950e06d7bfff4259972ae1312e8f913a, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.776846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8e78e-8bb6-439e-9ea7-5e9d6c544627 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.780632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 976/1962] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:03.782636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.804895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 987/1963] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:03 2022] PUT /identity/v3/projects/60b016d4443142d4bb81dae536748cf1/users/950e06d7bfff4259972ae1312e8f913a/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:03.806712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.808013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.808759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.811738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8154e8-fae2-4882-b588-71061e580f57 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.831957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:03.840290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:03.843042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.901782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.902642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.903103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.903510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.904458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.907336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:03.920117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 977/1964] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:03.924542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.939087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aced53-2e93-4509-868d-cc9ec68c1370 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:03.948788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 988/1965] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:03.953647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c68552-4fce-4306-a454-d8a4a53b5bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:03.954345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c68552-4fce-4306-a454-d8a4a53b5bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:03.954629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c68552-4fce-4306-a454-d8a4a53b5bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:03.954910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c68552-4fce-4306-a454-d8a4a53b5bb1 admin admin] PATH_INFO: `/v3/projects/1616ab7beef7499280c47114208878fe/users/eeb6f39c8c2b4a2f9ae77a20a20b655c/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:03.962731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:03.964164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:03.978554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c68552-4fce-4306-a454-d8a4a53b5bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1616ab7beef7499280c47114208878fe, user_id=eeb6f39c8c2b4a2f9ae77a20a20b655c, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:03.981349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c68552-4fce-4306-a454-d8a4a53b5bb1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:04.000348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:04.009018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 978/1966] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:03 2022] PUT /identity/v3/projects/1616ab7beef7499280c47114208878fe/users/eeb6f39c8c2b4a2f9ae77a20a20b655c/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:04.015120 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd4cf-cb02-49bc-b65c-dfb1b6d41305 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:04.015622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd4cf-cb02-49bc-b65c-dfb1b6d41305 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:04.016089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bd4cf-cb02-49bc-b65c-dfb1b6d41305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:04.040420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1a7bd4cf-cb02-49bc-b65c-dfb1b6d41305 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:04.042584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1a7bd4cf-cb02-49bc-b65c-dfb1b6d41305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:04.065378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df887864-5c91-4561-bfba-30a031a58b32 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:04.117149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b04604-aa03-420d-b130-21eebaaf2308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:04.117763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04604-aa03-420d-b130-21eebaaf2308 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:04.118064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04604-aa03-420d-b130-21eebaaf2308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:04.118334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b04604-aa03-420d-b130-21eebaaf2308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:04.119001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 979/1967] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:04.128130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b04604-aa03-420d-b130-21eebaaf2308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:04.128974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bd4cf-cb02-49bc-b65c-dfb1b6d41305 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:04.129620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b04604-aa03-420d-b130-21eebaaf2308 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:04.139149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bd4cf-cb02-49bc-b65c-dfb1b6d41305 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:04.140815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 989/1968] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:04.142833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:04.143442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:04.143717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:04.143983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:04.153671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:04.155913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:04.166917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 980/1969] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:05.460884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9cf6d-bce8-4be1-af50-d2ad2c94dcb5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:05.461404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9cf6d-bce8-4be1-af50-d2ad2c94dcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:05.461404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9cf6d-bce8-4be1-af50-d2ad2c94dcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:05.506848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-99d9cf6d-bce8-4be1-af50-d2ad2c94dcb5 None None] MFA Rules not processed for user `eeb6f39c8c2b4a2f9ae77a20a20b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:05.508169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-99d9cf6d-bce8-4be1-af50-d2ad2c94dcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:05.597669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:05.644031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 990/1970] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:05.656945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:05.658513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:05.732132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18229486-c551-4d34-90fc-7d12705c489d service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:05.737846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:05.738131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:05.738917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:05.739364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:05.751564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:05.753223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:05.765808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 981/1971] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:07.309780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bd4e2-daf2-40f5-9957-9f35a0d45def None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:07.310253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bd4e2-daf2-40f5-9957-9f35a0d45def None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:07.310342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bd4e2-daf2-40f5-9957-9f35a0d45def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:07.340075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-572bd4e2-daf2-40f5-9957-9f35a0d45def None None] MFA Rules not processed for user `eeb6f39c8c2b4a2f9ae77a20a20b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:07.341287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-572bd4e2-daf2-40f5-9957-9f35a0d45def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:07.349665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 991/1972] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:09.968911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:09.970369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:09.999545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331066bb-4aea-4305-9a1e-4793123de3d0 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:10.018529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:10.019684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:10.019798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:10.020051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] PATH_INFO: `/v3/users/6ac82d34dc09412593fc3940d873a2c7` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:10.021350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac82d34dc09412593fc3940d873a2c7)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:10.023282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:10.130492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac82d34dc09412593fc3940d873a2c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:10.131772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac82d34dc09412593fc3940d873a2c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:10.150606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac82d34dc09412593fc3940d873a2c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:10.175842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac82d34dc09412593fc3940d873a2c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:10.200254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac82d34dc09412593fc3940d873a2c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:10.219498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 982/1973] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:09 2022] DELETE /identity/v3/users/6ac82d34dc09412593fc3940d873a2c7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:10.222180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9c4e6-4941-4398-ba02-e9ed7b3a7651 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:10.222703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9c4e6-4941-4398-ba02-e9ed7b3a7651 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:10.222703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9c4e6-4941-4398-ba02-e9ed7b3a7651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:10.286024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-32e9c4e6-4941-4398-ba02-e9ed7b3a7651 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:10.287375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-32e9c4e6-4941-4398-ba02-e9ed7b3a7651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:10.383900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 992/1974] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:10.402656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:10.405408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:10.413806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:10.573323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bbc7e-3f33-4091-b40f-c22caf79f418 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:10.578941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:10.579859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:10.580236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:10.580611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:10.599560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:10.601487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:10.617244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 983/1975] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:11.256078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:11.256653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:11.256653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:11.290723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] MFA Rules not processed for user `950e06d7bfff4259972ae1312e8f913a`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:11.292110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:11.346529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:11.348962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:11.372405 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae831a-4be4-4c15-ac3a-7339b529bda0 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:11.382399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:11.383443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:11.384003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:11.385026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] PATH_INFO: `/v3/projects/6dcad8c29dc04887a3174a874bd7c023` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:11.387448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dcad8c29dc04887a3174a874bd7c023)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:11.391078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:11.419765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 993/1976] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:11.435535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:11.437727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:11.454259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b33f7e-960b-4fb7-8481-bd93ca4fff18 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:11.468952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dcad8c29dc04887a3174a874bd7c023', 'request_id': 'req-548734e4-86a0-438d-abcd-2dc9a49202b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.479530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dcad8c29dc04887a3174a874bd7c023', 'request_id': 'req-548734e4-86a0-438d-abcd-2dc9a49202b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.487951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dcad8c29dc04887a3174a874bd7c023', 'request_id': 'req-548734e4-86a0-438d-abcd-2dc9a49202b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.501128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] The token cache is being invalidate because project 6dcad8c29dc04887a3174a874bd7c023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:11.501505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.502495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.504499 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 984/1977] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:11 2022] DELETE /identity/v3/projects/6dcad8c29dc04887a3174a874bd7c023 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:11.507994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:11.509646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:11.539474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:11.540457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:11.541288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:11.541703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:11.542641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:11.545092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:11.598629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548734e4-86a0-438d-abcd-2dc9a49202b4 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:11.600694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 994/1978] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:11 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:11.604787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:11.606710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:11.614396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:11.657857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:11.688341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:11.689629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:11.690072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:11.690914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] PATH_INFO: `/v3/users/ad79c4ff515e4af2a610d89945f1ab4b` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:11.692240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad79c4ff515e4af2a610d89945f1ab4b)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:11.694266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:11.694598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82e499-22a7-4770-992a-a9eff12546bf admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:11.706695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:11.707427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:11.707792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:11.708130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:11.708875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:11.711272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:11.738209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 995/1979] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:11 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:11.741821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:11.752555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdacd-ceb9-40b1-8547-e8180b291cc5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:11.754160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad79c4ff515e4af2a610d89945f1ab4b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.755278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad79c4ff515e4af2a610d89945f1ab4b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.757881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:11.758572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:11.758881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:11.759273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:11.759924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:11.761818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:11.764706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad79c4ff515e4af2a610d89945f1ab4b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.777647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 996/1980] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:11.778661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:11.780298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad79c4ff515e4af2a610d89945f1ab4b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.781168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:11.788692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad79c4ff515e4af2a610d89945f1ab4b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:11.794919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe4a39-773b-405e-8555-14df574c6a94 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:11.796825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 985/1981] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:11 2022] DELETE /identity/v3/users/ad79c4ff515e4af2a610d89945f1ab4b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:11.882642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:11.884027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:11.884515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:11.884891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] PATH_INFO: `/v3/projects/85c573761eb54bd5bfe7e8a9a0088041/users/28e6345f277e45a7b9a3636737c08b29/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:11.904406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c573761eb54bd5bfe7e8a9a0088041, user_id=28e6345f277e45a7b9a3636737c08b29, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:11.907634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:11.922374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 997/1982] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:11 2022] PUT /identity/v3/projects/85c573761eb54bd5bfe7e8a9a0088041/users/28e6345f277e45a7b9a3636737c08b29/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:11.926574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:11.937514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e0b8d-099a-40d2-b72d-d4e2e6ec8f22 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:12.026803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:12.027503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:12.027745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:12.028019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:12.028786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:12.030685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:12.042951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 986/1983] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:12.047008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:12.068223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227d98e-5e23-4444-bb39-b967ba8a9e6d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:12.099085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acd147-4bc7-490f-958d-0fc9b22dba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:12.099782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd147-4bc7-490f-958d-0fc9b22dba50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:12.100063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd147-4bc7-490f-958d-0fc9b22dba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:12.100336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd147-4bc7-490f-958d-0fc9b22dba50 admin admin] PATH_INFO: `/v3/projects/85c573761eb54bd5bfe7e8a9a0088041/users/28e6345f277e45a7b9a3636737c08b29/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:12.105247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acd147-4bc7-490f-958d-0fc9b22dba50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c573761eb54bd5bfe7e8a9a0088041, user_id=28e6345f277e45a7b9a3636737c08b29, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:12.108977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acd147-4bc7-490f-958d-0fc9b22dba50 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:12.148521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 998/1984] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:12 2022] PUT /identity/v3/projects/85c573761eb54bd5bfe7e8a9a0088041/users/28e6345f277e45a7b9a3636737c08b29/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:12.171104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:12.173560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:12.200132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d7477-4284-4fd1-a708-28a8cea9c412 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:12.290794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:12.322257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:12.323070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:12.323344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:12.323610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] PATH_INFO: `/v3/projects/603b0b72733d4ccfbf98b75be9610a46` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:12.325536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=603b0b72733d4ccfbf98b75be9610a46)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:12.328947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:12.400181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603b0b72733d4ccfbf98b75be9610a46', 'request_id': 'req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:12.411042 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603b0b72733d4ccfbf98b75be9610a46', 'request_id': 'req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:12.420493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603b0b72733d4ccfbf98b75be9610a46', 'request_id': 'req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:12.438767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] The token cache is being invalidate because project 603b0b72733d4ccfbf98b75be9610a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:12.439300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:12.440230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9b3dcb34-ccf0-404b-ba8f-0aebb7e48e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:12.445558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 987/1985] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:12 2022] DELETE /identity/v3/projects/603b0b72733d4ccfbf98b75be9610a46 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:12.447361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ec9be-52b3-414b-b105-36e10f5c5800 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:12.447876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ec9be-52b3-414b-b105-36e10f5c5800 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:12.448035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ec9be-52b3-414b-b105-36e10f5c5800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:12.503217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-086ec9be-52b3-414b-b105-36e10f5c5800 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:12.527201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-086ec9be-52b3-414b-b105-36e10f5c5800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:12.637572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:12.658854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 999/1986] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:12.670290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f97a1a-5ca3-46ee-b2cb-02eb0342f674 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:12.670555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f97a1a-5ca3-46ee-b2cb-02eb0342f674 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:12.671116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f97a1a-5ca3-46ee-b2cb-02eb0342f674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:12.732853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-77f97a1a-5ca3-46ee-b2cb-02eb0342f674 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:12.734807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-77f97a1a-5ca3-46ee-b2cb-02eb0342f674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:12.748864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 988/1987] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:12.753042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ec9be-52b3-414b-b105-36e10f5c5800 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:12.772078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ec9be-52b3-414b-b105-36e10f5c5800 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:12.780471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:12.780471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:12.780471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:12.780856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:12.781569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:12.781877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:12.784809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:12.799055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1000/1988] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:12.803149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f97a1a-5ca3-46ee-b2cb-02eb0342f674 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:12.814984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f97a1a-5ca3-46ee-b2cb-02eb0342f674 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:12.821375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:12.822380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:12.822664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:12.822932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:12.823693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:12.825897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:12.871785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 989/1989] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:12 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:12.873405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:12.889833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d5c15-c8b5-4a5d-a964-559bfd330a54 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:13.026144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:13.027473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:13.027752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:13.028027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:13.029256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:13.032446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:13.110279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1001/1990] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:12 2022] POST /identity/v3/users => generated 284 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:13.115412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:13.144092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4444ed-8a1f-434f-9ec4-83f753425fde admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:13.158391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:13.159241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:13.159657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:13.160058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:13.163366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:13.165207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:13.207205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 990/1991] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:13.211768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:13.228612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74deb2d-c9c3-48e7-ae6e-e9fa3c0bd081 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:13.242465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d8e8f-610f-4fcd-bdad-969acb09b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:13.243100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d8e8f-610f-4fcd-bdad-969acb09b54d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:13.243377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d8e8f-610f-4fcd-bdad-969acb09b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:13.243646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d8e8f-610f-4fcd-bdad-969acb09b54d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:13.244608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d8e8f-610f-4fcd-bdad-969acb09b54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:13.251108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d8e8f-610f-4fcd-bdad-969acb09b54d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:13.280977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1002/1992] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:13.286218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:13.302454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc96ff-60ca-43d7-8c96-227c8b178753 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:13.315474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:13.317223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:13.319910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:13.322321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] PATH_INFO: `/v3/projects/95604af0d2f34b6d88bc871d902efd59/users/5def6faf68d64b21bfbb934aa222ce62/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:13.360616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] RBAC: Authorizing `identity:create_grant(project_id=95604af0d2f34b6d88bc871d902efd59, user_id=5def6faf68d64b21bfbb934aa222ce62, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:13.363710 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:13.399441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 991/1993] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:13 2022] PUT /identity/v3/projects/95604af0d2f34b6d88bc871d902efd59/users/5def6faf68d64b21bfbb934aa222ce62/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:13.413362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa4062-728f-45d1-b5e6-ca51562a5515 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:13.413885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa4062-728f-45d1-b5e6-ca51562a5515 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:13.414311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa4062-728f-45d1-b5e6-ca51562a5515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:13.491392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b6fa4062-728f-45d1-b5e6-ca51562a5515 None None] MFA Rules not processed for user `5def6faf68d64b21bfbb934aa222ce62`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:13.492640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b6fa4062-728f-45d1-b5e6-ca51562a5515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:13.653978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1003/1994] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:13.658864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:13.809996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb59373-040f-45fa-bc2e-2f15a12b467b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:13.809996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:13.810677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:13.810677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:13.811925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:13.813776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:13.815869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:13.869308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 992/1995] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:13 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:13.875425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa4062-728f-45d1-b5e6-ca51562a5515 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:13.889440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa4062-728f-45d1-b5e6-ca51562a5515 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:13.986451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:14.039856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:14.040548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:14.040877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:14.046078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:14.050322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:14.056321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:14.113721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:14.113929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1004/1996] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:13 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:14.125408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:14.149367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba187b2-232c-41c4-8071-0a789e745611 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:14.172804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:14.173464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:14.173876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:14.174157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:14.175132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:14.182593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:14.221528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 993/1997] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:14.225481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:14.240645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ed267-ec28-4e96-8073-059206ab5c97 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:14.255360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:14.258012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:14.258300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:14.258579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] PATH_INFO: `/v3/projects/483052d840604a85b588322b68afdf92/users/9044080b088043c78a903d179320c83f/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:14.327283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483052d840604a85b588322b68afdf92, user_id=9044080b088043c78a903d179320c83f, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:14.338117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:14.383158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1005/1998] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:14 2022] PUT /identity/v3/projects/483052d840604a85b588322b68afdf92/users/9044080b088043c78a903d179320c83f/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:14.386847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:14.402754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70f91a-92ad-429b-9a3d-24e1650e5942 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:14.505459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45985928-b050-4ded-a4a2-db4dfa3f600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:14.507812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45985928-b050-4ded-a4a2-db4dfa3f600f admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:14.507948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45985928-b050-4ded-a4a2-db4dfa3f600f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:14.508788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45985928-b050-4ded-a4a2-db4dfa3f600f admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:14.510026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45985928-b050-4ded-a4a2-db4dfa3f600f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:14.512416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45985928-b050-4ded-a4a2-db4dfa3f600f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:14.533556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 994/1999] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:14.534543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:14.561427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d5d37-5f21-40e6-8f0f-c0b0f39814a5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:14.579022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:14.579724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:14.580012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:14.580289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] PATH_INFO: `/v3/projects/483052d840604a85b588322b68afdf92/users/9044080b088043c78a903d179320c83f/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:14.586327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483052d840604a85b588322b68afdf92, user_id=9044080b088043c78a903d179320c83f, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:14.589206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:14.615623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1006/2000] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:14 2022] PUT /identity/v3/projects/483052d840604a85b588322b68afdf92/users/9044080b088043c78a903d179320c83f/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:14.628790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:14.629152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:14.629491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:14.696682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] MFA Rules not processed for user `9044080b088043c78a903d179320c83f`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:14.698841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:14.803659 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 995/2001] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:14.816064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:14.817922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:14.934676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f397-37e7-4f40-817f-342c852bddc8 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:14.946031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:14.947050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:14.947581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:14.948055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:14.961990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:14.963960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:14.980733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1007/2002] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:15.094622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:15.096622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:15.119130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1553a9-2e5a-4879-aef2-1b37f604aba2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:15.137575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc00932-c052-484f-8844-5118f60ab6f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:15.138407 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc00932-c052-484f-8844-5118f60ab6f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:15.138607 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc00932-c052-484f-8844-5118f60ab6f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:15.139632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc00932-c052-484f-8844-5118f60ab6f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:15.319826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc00932-c052-484f-8844-5118f60ab6f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:15.321876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc00932-c052-484f-8844-5118f60ab6f6 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:15.352326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 996/2003] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:19.492526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:19.697594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59750667-cc7b-4fab-b476-d1fe2ff8a7de service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:19.715440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:19.716505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:19.716901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:19.717229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] PATH_INFO: `/v3/users/5def6faf68d64b21bfbb934aa222ce62` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:19.719035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5def6faf68d64b21bfbb934aa222ce62)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:19.722899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:19.806720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:19.809455 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5def6faf68d64b21bfbb934aa222ce62'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:19.827758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5def6faf68d64b21bfbb934aa222ce62'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:19.846589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5def6faf68d64b21bfbb934aa222ce62'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:19.855132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5def6faf68d64b21bfbb934aa222ce62'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:19.855954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5def6faf68d64b21bfbb934aa222ce62'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:19.866287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1008/2004] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:19 2022] DELETE /identity/v3/users/5def6faf68d64b21bfbb934aa222ce62 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:19.872563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:19.873688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:19.873795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:19.898849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:19.900107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:19.905626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:19.984853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 997/2005] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:20.000427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:20.005645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:20.098292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2b0dc-48aa-4001-a7e0-0f074dfb0a50 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:20.103257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:20.104245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:20.104706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:20.105247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:20.119993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:20.122816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:20.140442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1009/2006] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:21.224577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:21.226267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:21.243216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff20598-a83d-4829-b087-48eca17062f0 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:21.249397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:21.250185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:21.250466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:21.250736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] PATH_INFO: `/v3/projects/95604af0d2f34b6d88bc871d902efd59` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:21.253521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=95604af0d2f34b6d88bc871d902efd59)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:21.256522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:21.340633 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95604af0d2f34b6d88bc871d902efd59', 'request_id': 'req-140141eb-8b19-4029-bc64-d21404413a3c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.353228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95604af0d2f34b6d88bc871d902efd59', 'request_id': 'req-140141eb-8b19-4029-bc64-d21404413a3c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.378842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95604af0d2f34b6d88bc871d902efd59', 'request_id': 'req-140141eb-8b19-4029-bc64-d21404413a3c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.400918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] The token cache is being invalidate because project 95604af0d2f34b6d88bc871d902efd59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:21.401113 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.403126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.407337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 998/2007] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:21 2022] DELETE /identity/v3/projects/95604af0d2f34b6d88bc871d902efd59 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:21.411363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:21.424821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e0909-a495-45a3-a671-3977f405693e service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:21.555449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:21.556123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:21.556401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:21.556668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] PATH_INFO: `/v3/users/9044080b088043c78a903d179320c83f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:21.557871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9044080b088043c78a903d179320c83f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:21.559984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:21.673147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9044080b088043c78a903d179320c83f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.692331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9044080b088043c78a903d179320c83f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.715928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:21.724556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9044080b088043c78a903d179320c83f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.737813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9044080b088043c78a903d179320c83f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.739197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0fb636f2-905b-4602-8664-f003ed85a4c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9044080b088043c78a903d179320c83f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:21.747826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1010/2008] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:21 2022] DELETE /identity/v3/users/9044080b088043c78a903d179320c83f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:22.785815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:22.787297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:22.831254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140141eb-8b19-4029-bc64-d21404413a3c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:22.976985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:22.978789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:22.979305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:22.980133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] PATH_INFO: `/v3/projects/483052d840604a85b588322b68afdf92` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:22.981812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] RBAC: Authorizing `identity:delete_project(project_id=483052d840604a85b588322b68afdf92)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:22.985919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:23.055959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:23.069025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483052d840604a85b588322b68afdf92', 'request_id': 'req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:23.079238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483052d840604a85b588322b68afdf92', 'request_id': 'req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:23.087265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483052d840604a85b588322b68afdf92', 'request_id': 'req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:23.102357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] The token cache is being invalidate because project 483052d840604a85b588322b68afdf92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:23.104990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:23.105942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cbb9a6d5-0fee-42e1-b72d-10b18f2bbefb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:23.108126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 999/2009] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:22 2022] DELETE /identity/v3/projects/483052d840604a85b588322b68afdf92 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:23.115641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea18d-9607-4831-a020-b264d478dd80 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.116129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea18d-9607-4831-a020-b264d478dd80 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.116233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fea18d-9607-4831-a020-b264d478dd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.138768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-30fea18d-9607-4831-a020-b264d478dd80 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:23.140124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-30fea18d-9607-4831-a020-b264d478dd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:23.209802 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1011/2010] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:23.218586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3583-a376-4ce1-a846-b92230ba466a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.218898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3583-a376-4ce1-a846-b92230ba466a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.219172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed3583-a376-4ce1-a846-b92230ba466a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.241524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-92ed3583-a376-4ce1-a846-b92230ba466a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:23.243404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-92ed3583-a376-4ce1-a846-b92230ba466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:23.252167 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.253639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.254028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.261857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1000/2011] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:23.262728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed3583-a376-4ce1-a846-b92230ba466a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:23.273133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed3583-a376-4ce1-a846-b92230ba466a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:23.278951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:23.280384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.280666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.281003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.281664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:23.281964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:23.284616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:23.327069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1001/2012] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:23.344846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:23.381414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae011233-553d-4392-9ad4-d65b09f38652 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:23.385586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] MFA Rules not processed for user `28e6345f277e45a7b9a3636737c08b29`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:23.386825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:23.407460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:23.408312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.408770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.409198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.410159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:23.414171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:23.493282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1002/2013] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:23 2022] POST /identity/v3/projects => generated 399 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:23.497775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:23.548523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1012/2014] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:23.564366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:23.566204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:23.617389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c41691-4c85-43cc-887d-75104dac1961 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:23.626417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:23.628046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.628411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.628808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.629678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:23.632355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:23.696360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1003/2015] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:23 2022] POST /identity/v3/users => generated 300 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:23.702053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:23.713701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd752-117f-4f54-8b7b-b3d11dd5e58e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:23.721681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:23.723222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.723510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.723781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.724905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:23.726796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:23.738160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1846b53-f07c-4ea3-8f96-319f028bf077 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:23.741541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1004/2016] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:23.745887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:23.755105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:23.765841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cad85-c529-4cc8-89bd-7e93524edd04 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:23.772446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:23.773263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.773645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.774018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.774826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:23.776558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:23.798179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1005/2017] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:23.802264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:23.815323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfa164-9162-48a8-869c-6fe8234698ad admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:23.823503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882ff61-2628-4320-833d-42dfe4e85fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:23.824420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ff61-2628-4320-833d-42dfe4e85fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.824829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ff61-2628-4320-833d-42dfe4e85fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.825202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ff61-2628-4320-833d-42dfe4e85fbd admin admin] PATH_INFO: `/v3/projects/2159330a7d4547929d82b528df3e6b19/users/c0aef615c94a4a129053d24e91e4ea9f/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.875685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882ff61-2628-4320-833d-42dfe4e85fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2159330a7d4547929d82b528df3e6b19, user_id=c0aef615c94a4a129053d24e91e4ea9f, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:23.878542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882ff61-2628-4320-833d-42dfe4e85fbd admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:23.893080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:23.893925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.894200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.894588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.901218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:23.908737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1006/2018] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:23 2022] PUT /identity/v3/projects/2159330a7d4547929d82b528df3e6b19/users/c0aef615c94a4a129053d24e91e4ea9f/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:23.924142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0053-c2c0-40e6-873e-7df8572c6a87 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:23.926977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0053-c2c0-40e6-873e-7df8572c6a87 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:23.928260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0053-c2c0-40e6-873e-7df8572c6a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:23.981316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e4cf0053-c2c0-40e6-873e-7df8572c6a87 None None] MFA Rules not processed for user `c0aef615c94a4a129053d24e91e4ea9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:23.984091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e4cf0053-c2c0-40e6-873e-7df8572c6a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:24.004831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:24.019186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:24.023649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:24.040357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1013/2019] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:24.115282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1007/2020] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:24.117928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:24.134854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35baf-6236-4ec0-8e12-dd15e66bf1aa service nova] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:24.284568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:24.285670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:24.285951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:24.286268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:24.286958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:24.289049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:24.348621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1014/2021] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:24 2022] POST /identity/v3/projects => generated 401 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:24.352478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf0053-c2c0-40e6-873e-7df8572c6a87 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:24.371860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf0053-c2c0-40e6-873e-7df8572c6a87 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:24.456203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:24.457252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:24.457622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:24.457991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:24.459001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:24.461494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:24.500863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1008/2022] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:24 2022] POST /identity/v3/users => generated 307 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:24.505822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:24.522237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cfe22-4bf0-48af-9b14-2abb6be2895b admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:24.528263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:24.530106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:24.530894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:24.531247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:24.531980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:24.534350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:24.552101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1015/2023] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:24.557171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:24.571557 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4feac1-9e00-4fdf-9539-1ad81feaa444 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:24.583763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:24.584604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:24.585355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:24.585861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] PATH_INFO: `/v3/projects/359e0e13503142fcb4ad492ca4c675b9/users/4436d4c869ed4e6687875193789552ec/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:24.616198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359e0e13503142fcb4ad492ca4c675b9, user_id=4436d4c869ed4e6687875193789552ec, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:24.659633 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:24.697518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1009/2024] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:24 2022] PUT /identity/v3/projects/359e0e13503142fcb4ad492ca4c675b9/users/4436d4c869ed4e6687875193789552ec/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:24.702663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:24.732675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a81d3-5ea0-48f1-b98a-ddba2c17b763 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:24.782582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:24.917892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f945b-fc97-482d-814d-0062e12ed352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:24.918805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f945b-fc97-482d-814d-0062e12ed352 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:24.919223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f945b-fc97-482d-814d-0062e12ed352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:24.920258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f945b-fc97-482d-814d-0062e12ed352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:24.921301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f945b-fc97-482d-814d-0062e12ed352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:24.923627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f945b-fc97-482d-814d-0062e12ed352 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:24.939353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1016/2025] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:24.943603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:24.956809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68db7f-b04b-4a74-9e31-054610dbbec7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:24.963444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:24.964362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:24.964861 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:24.965274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] PATH_INFO: `/v3/projects/359e0e13503142fcb4ad492ca4c675b9/users/4436d4c869ed4e6687875193789552ec/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:24.969609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=359e0e13503142fcb4ad492ca4c675b9, user_id=4436d4c869ed4e6687875193789552ec, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:24.973802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:24.990003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1010/2026] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:24 2022] PUT /identity/v3/projects/359e0e13503142fcb4ad492ca4c675b9/users/4436d4c869ed4e6687875193789552ec/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:25.001452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd32d-d90a-4d8c-9aac-c823fee99cbf None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:25.002029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd32d-d90a-4d8c-9aac-c823fee99cbf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:25.002421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd32d-d90a-4d8c-9aac-c823fee99cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:25.055819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ba1cd32d-d90a-4d8c-9aac-c823fee99cbf None None] MFA Rules not processed for user `4436d4c869ed4e6687875193789552ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:25.057434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ba1cd32d-d90a-4d8c-9aac-c823fee99cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:25.223920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1017/2027] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:25.236222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:25.239429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:25.343571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602d5ad-43ff-4b3d-94d2-40f60483e3c1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:25.350440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf62cc-1fac-47a0-bce1-a94e3419eba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:25.351520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf62cc-1fac-47a0-bce1-a94e3419eba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:25.351903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf62cc-1fac-47a0-bce1-a94e3419eba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:25.352278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf62cc-1fac-47a0-bce1-a94e3419eba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:25.364916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf62cc-1fac-47a0-bce1-a94e3419eba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:25.367223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf62cc-1fac-47a0-bce1-a94e3419eba4 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:25.409921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1011/2028] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:25 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:25.512435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cd32d-d90a-4d8c-9aac-c823fee99cbf None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:25.526074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cd32d-d90a-4d8c-9aac-c823fee99cbf None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:25.533327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:25.534296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:25.534721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:25.535137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:25.677352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:25.678938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:25.691956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1018/2029] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:26.664960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d41447-6a9e-4d5a-98b2-cd13334112cd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:26.666100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d41447-6a9e-4d5a-98b2-cd13334112cd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:26.666593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d41447-6a9e-4d5a-98b2-cd13334112cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:26.734987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-56d41447-6a9e-4d5a-98b2-cd13334112cd None None] MFA Rules not processed for user `950e06d7bfff4259972ae1312e8f913a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:26.736689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-56d41447-6a9e-4d5a-98b2-cd13334112cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:26.754169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:26.855247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1012/2030] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:26.965452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:26.968447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:27.079236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:27.109764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d979f-d733-46b6-a962-c553aa4b3365 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:27.114640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:27.115293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:27.115572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:27.115851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] PATH_INFO: `/v3/users/105c6710d8994b0c9a1b66c4dad6dbc8` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:27.117123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=105c6710d8994b0c9a1b66c4dad6dbc8)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:27.119051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:27.203974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105c6710d8994b0c9a1b66c4dad6dbc8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.214638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105c6710d8994b0c9a1b66c4dad6dbc8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.246083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105c6710d8994b0c9a1b66c4dad6dbc8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.262450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105c6710d8994b0c9a1b66c4dad6dbc8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.263799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-0a658f89-9365-4e2e-bb72-e6490ace6115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105c6710d8994b0c9a1b66c4dad6dbc8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.282489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1019/2031] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:26 2022] DELETE /identity/v3/users/105c6710d8994b0c9a1b66c4dad6dbc8 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:27.527814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d41447-6a9e-4d5a-98b2-cd13334112cd None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:27.544151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d41447-6a9e-4d5a-98b2-cd13334112cd None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:27.665627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:27.668108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:27.668766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:27.669155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] PATH_INFO: `/v3/projects/943f7e5ccad5410896a40678aeaaf1c1` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:27.670737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] RBAC: Authorizing `identity:delete_project(project_id=943f7e5ccad5410896a40678aeaaf1c1)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:27.673054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:27.734242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943f7e5ccad5410896a40678aeaaf1c1', 'request_id': 'req-5cb0133a-0af9-4f33-9b53-688710de545c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.741153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943f7e5ccad5410896a40678aeaaf1c1', 'request_id': 'req-5cb0133a-0af9-4f33-9b53-688710de545c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.746686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943f7e5ccad5410896a40678aeaaf1c1', 'request_id': 'req-5cb0133a-0af9-4f33-9b53-688710de545c'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.758125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] The token cache is being invalidate because project 943f7e5ccad5410896a40678aeaaf1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:27.758413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.759047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5cb0133a-0af9-4f33-9b53-688710de545c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:27.760538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1013/2032] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:27 2022] DELETE /identity/v3/projects/943f7e5ccad5410896a40678aeaaf1c1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:27.765619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ead75-4d3d-4e51-9eff-ad7d3ecd9b31 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:27.765903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ead75-4d3d-4e51-9eff-ad7d3ecd9b31 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:27.766170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ead75-4d3d-4e51-9eff-ad7d3ecd9b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:27.786292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-529ead75-4d3d-4e51-9eff-ad7d3ecd9b31 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:27.787492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-529ead75-4d3d-4e51-9eff-ad7d3ecd9b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:27.863445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1020/2033] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:27.865391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc97fd-59da-4023-a54f-120303fd04ec None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:27.865769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc97fd-59da-4023-a54f-120303fd04ec None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:27.866054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc97fd-59da-4023-a54f-120303fd04ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:27.888082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-f1fc97fd-59da-4023-a54f-120303fd04ec None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:27.890065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f1fc97fd-59da-4023-a54f-120303fd04ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:27.899849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1014/2034] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:27.904654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ead75-4d3d-4e51-9eff-ad7d3ecd9b31 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:27.920246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ead75-4d3d-4e51-9eff-ad7d3ecd9b31 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:27.925129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:27.925733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:27.926007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:27.926272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:27.926919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:27.927329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:27.928989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:27.944931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1021/2035] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:27.946135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc97fd-59da-4023-a54f-120303fd04ec None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:27.960036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc97fd-59da-4023-a54f-120303fd04ec None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:27.965085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:27.965815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:27.966247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:27.967069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:27.968050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:27.970221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.002605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1015/2036] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:27 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:28.008203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.021486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e31fae-bfdc-4651-8d79-04fa4939f826 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.072442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:28.089482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.090066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.090344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.090610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.091241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.095217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.119499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1022/2037] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:28 2022] POST /identity/v3/users => generated 303 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:28.121322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.134812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745da8d-b8a0-4815-840c-30f53ce49ba6 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.140965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.141727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.142161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.142502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.143332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.145692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.164558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1016/2038] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:28.168780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.180457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659132d-b0e4-41af-b03b-8bc9112cb5ae admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.185866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38437757-347d-419a-98af-ae50a35063a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.186701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437757-347d-419a-98af-ae50a35063a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.187001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437757-347d-419a-98af-ae50a35063a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.187287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437757-347d-419a-98af-ae50a35063a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.187938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38437757-347d-419a-98af-ae50a35063a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.189595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38437757-347d-419a-98af-ae50a35063a8 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.199821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1023/2039] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:28.204872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.215206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb245ce0-6d99-4d8d-884b-0b81318fab2b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.221093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.221777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.222146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.222387 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] PATH_INFO: `/v3/projects/bd29944da180460b90e4596be63c9605/users/63d349a3d2394796a5ba3bf2ff198395/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.225123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:28.241827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd29944da180460b90e4596be63c9605, user_id=63d349a3d2394796a5ba3bf2ff198395, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.244812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.258590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1017/2040] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:28 2022] PUT /identity/v3/projects/bd29944da180460b90e4596be63c9605/users/63d349a3d2394796a5ba3bf2ff198395/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:28.272016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0990332-8b64-45bf-9d37-a82f69c73d42 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.272301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0990332-8b64-45bf-9d37-a82f69c73d42 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.272576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0990332-8b64-45bf-9d37-a82f69c73d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.303251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c0990332-8b64-45bf-9d37-a82f69c73d42 None None] MFA Rules not processed for user `63d349a3d2394796a5ba3bf2ff198395`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:28.304471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c0990332-8b64-45bf-9d37-a82f69c73d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:28.376514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1024/2041] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:28 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:28.380982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.458781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432b40-ba74-48ab-a5dd-6dfa3c6d63ad admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.463620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.464825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.464825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.464825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.465476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.468193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.500888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1018/2042] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:28 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:28.502732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0990332-8b64-45bf-9d37-a82f69c73d42 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.515104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0990332-8b64-45bf-9d37-a82f69c73d42 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.580962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.582098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.582543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.582965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.583755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.586249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.614618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1025/2043] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:28 2022] POST /identity/v3/users => generated 308 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:28.618354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.633009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b564e-6bd5-4c4b-b62b-3c07a5afa8be admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.641615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.642249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.642548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.642789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.645265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.646884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.660296 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1019/2044] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:28.664182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.675301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7215b1-d685-4a07-9339-344084f560da admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.680583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.681344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.681752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.682013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] PATH_INFO: `/v3/projects/19f7574cc86a4069b7b1b1f3143cbc09/users/f7aca695ccf54970a121d767ab6ad0b4/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.705309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f7574cc86a4069b7b1b1f3143cbc09, user_id=f7aca695ccf54970a121d767ab6ad0b4, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.708745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.727967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1026/2045] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:28 2022] PUT /identity/v3/projects/19f7574cc86a4069b7b1b1f3143cbc09/users/f7aca695ccf54970a121d767ab6ad0b4/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:28.731827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.746150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe595c6-35e7-4d97-b025-2ea5de597224 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.821306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ce850-2385-4016-8fc1-0ad97b7403e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.821914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ce850-2385-4016-8fc1-0ad97b7403e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.822185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ce850-2385-4016-8fc1-0ad97b7403e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.822452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ce850-2385-4016-8fc1-0ad97b7403e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.823091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ce850-2385-4016-8fc1-0ad97b7403e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.824926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ce850-2385-4016-8fc1-0ad97b7403e2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.848919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1020/2046] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:28.852412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:28.867013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd7032-15fd-4a9a-b5b1-0465ca474417 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:28.879005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:28.879620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.879899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.880173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] PATH_INFO: `/v3/projects/19f7574cc86a4069b7b1b1f3143cbc09/users/f7aca695ccf54970a121d767ab6ad0b4/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.883735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f7574cc86a4069b7b1b1f3143cbc09, user_id=f7aca695ccf54970a121d767ab6ad0b4, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:28.886486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:28.897753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1027/2047] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:28 2022] PUT /identity/v3/projects/19f7574cc86a4069b7b1b1f3143cbc09/users/f7aca695ccf54970a121d767ab6ad0b4/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:28.909763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365913b8-a1f4-4fb5-81cd-9931b4d12658 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:28.910045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365913b8-a1f4-4fb5-81cd-9931b4d12658 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:28.910315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365913b8-a1f4-4fb5-81cd-9931b4d12658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:28.943005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-365913b8-a1f4-4fb5-81cd-9931b4d12658 None None] MFA Rules not processed for user `f7aca695ccf54970a121d767ab6ad0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:28.944529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-365913b8-a1f4-4fb5-81cd-9931b4d12658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:29.009941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:29.040437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1021/2048] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:28 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:29.144937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:29.148180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:29.166364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:29.247900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1f5b7-c474-428a-8c3f-290c1f6e64eb admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:29.253069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbe2b6-8fab-4dca-9591-79f337777a27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:29.253972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe2b6-8fab-4dca-9591-79f337777a27 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:29.254331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe2b6-8fab-4dca-9591-79f337777a27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:29.254609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbe2b6-8fab-4dca-9591-79f337777a27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:29.281253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbe2b6-8fab-4dca-9591-79f337777a27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:29.282845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfbe2b6-8fab-4dca-9591-79f337777a27 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:29.294206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1028/2049] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:29 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:29.480459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365913b8-a1f4-4fb5-81cd-9931b4d12658 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:29.547632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365913b8-a1f4-4fb5-81cd-9931b4d12658 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:29.552200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:29.552853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:29.553135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:29.553550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] PATH_INFO: `/v3/users/63d349a3d2394796a5ba3bf2ff198395` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:29.554669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d349a3d2394796a5ba3bf2ff198395)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:29.556839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:29.608339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d349a3d2394796a5ba3bf2ff198395'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:29.609105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d349a3d2394796a5ba3bf2ff198395'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:29.618280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d349a3d2394796a5ba3bf2ff198395'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:29.631359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d349a3d2394796a5ba3bf2ff198395'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:29.639032 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d349a3d2394796a5ba3bf2ff198395'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:29.645182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1022/2050] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:29 2022] DELETE /identity/v3/users/63d349a3d2394796a5ba3bf2ff198395 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:29.649642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:29.649928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:29.650203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:29.669379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:29.670909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:29.736058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1029/2051] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:29.745727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:29.747262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:29.808463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fe132-66ad-4cf8-a638-efac0eafebf2 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:29.812906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:29.813511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:29.813785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:29.814054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:29.823100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:29.824529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:29.830089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:29.834946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1023/2052] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:30.112591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:30.114095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:30.126697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b34d5-5f07-4b0f-8316-27ec263709fe None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:30.131344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:30.131988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:30.132257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:30.132522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] PATH_INFO: `/v3/projects/bd29944da180460b90e4596be63c9605` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:30.133678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd29944da180460b90e4596be63c9605)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:30.135584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:30.187651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd29944da180460b90e4596be63c9605', 'request_id': 'req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.195159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd29944da180460b90e4596be63c9605', 'request_id': 'req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.201262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd29944da180460b90e4596be63c9605', 'request_id': 'req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.212306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] The token cache is being invalidate because project bd29944da180460b90e4596be63c9605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:30.212695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.213444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.214935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1030/2053] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:30 2022] DELETE /identity/v3/projects/bd29944da180460b90e4596be63c9605 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:30.219606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:30.229130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6850b65-5fd8-4ddc-88f4-6c75088d177d service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:30.303347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:30.304300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:30.304582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:30.304885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] PATH_INFO: `/v3/users/f7aca695ccf54970a121d767ab6ad0b4` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:30.306177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7aca695ccf54970a121d767ab6ad0b4)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:30.309305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:30.339087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:30.341464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:30.359633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89cb59-b8f5-4ae2-a5c7-11cc6921cc98 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:30.365089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:30.366177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:30.366476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:30.366755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] PATH_INFO: `/v3/users/c0aef615c94a4a129053d24e91e4ea9f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:30.367986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0aef615c94a4a129053d24e91e4ea9f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:30.370148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:30.408938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7aca695ccf54970a121d767ab6ad0b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.410370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:30.412357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7aca695ccf54970a121d767ab6ad0b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.432148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7aca695ccf54970a121d767ab6ad0b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.452420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7aca695ccf54970a121d767ab6ad0b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.454238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0aef615c94a4a129053d24e91e4ea9f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.461270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a9c24e38-2cef-42f6-bfa6-769f36908388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7aca695ccf54970a121d767ab6ad0b4'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.465105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0aef615c94a4a129053d24e91e4ea9f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.473214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1024/2054] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:30 2022] DELETE /identity/v3/users/f7aca695ccf54970a121d767ab6ad0b4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:30.497136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0aef615c94a4a129053d24e91e4ea9f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.513543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0aef615c94a4a129053d24e91e4ea9f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.514516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0aef615c94a4a129053d24e91e4ea9f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:30.524901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1031/2055] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:30 2022] DELETE /identity/v3/users/c0aef615c94a4a129053d24e91e4ea9f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:30.528261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:30.528544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:30.529347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:30.556666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:30.558516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:30.680132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1025/2056] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:30.701196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:30.702694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:30.815761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bc3f1-4cca-4156-a555-af9e7a759b44 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:30.832546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef287e5-6178-4225-af20-bade4b25b149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:30.833200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef287e5-6178-4225-af20-bade4b25b149 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:30.833484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef287e5-6178-4225-af20-bade4b25b149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:30.833743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef287e5-6178-4225-af20-bade4b25b149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:30.861056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef287e5-6178-4225-af20-bade4b25b149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:30.862642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef287e5-6178-4225-af20-bade4b25b149 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:30.880244 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1032/2057] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:30.921500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:30.923987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:30.945513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8bf6-313d-45c0-88cc-14ed439ee930 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:30.951624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:30.952507 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:30.952735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:30.953148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] PATH_INFO: `/v3/projects/19f7574cc86a4069b7b1b1f3143cbc09` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:30.954325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f7574cc86a4069b7b1b1f3143cbc09)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:30.956336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:30.977101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:31.020565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f7574cc86a4069b7b1b1f3143cbc09', 'request_id': 'req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.031517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f7574cc86a4069b7b1b1f3143cbc09', 'request_id': 'req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.037741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f7574cc86a4069b7b1b1f3143cbc09', 'request_id': 'req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.050451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] The token cache is being invalidate because project 19f7574cc86a4069b7b1b1f3143cbc09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:31.050824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.051414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3a6d1c51-e5ba-41b5-bdc4-912ec3d628a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.052942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1026/2058] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:30 2022] DELETE /identity/v3/projects/19f7574cc86a4069b7b1b1f3143cbc09 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:31.058106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.060855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.061207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.087755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:31.089225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:31.164410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1033/2059] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:31.172112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44b13e-fc4e-4f50-b9cb-e05cc8b8784a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.172398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44b13e-fc4e-4f50-b9cb-e05cc8b8784a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.172670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44b13e-fc4e-4f50-b9cb-e05cc8b8784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.201292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-bd44b13e-fc4e-4f50-b9cb-e05cc8b8784a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:31.202747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bd44b13e-fc4e-4f50-b9cb-e05cc8b8784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:31.207027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.208975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:31.212831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1027/2060] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:31.223492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44b13e-fc4e-4f50-b9cb-e05cc8b8784a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.227575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1261-7661-4190-b47f-6090fa467662 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.233607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.234269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.234603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.234906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] PATH_INFO: `/v3/projects/2159330a7d4547929d82b528df3e6b19` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.236105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2159330a7d4547929d82b528df3e6b19)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.238920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.246931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44b13e-fc4e-4f50-b9cb-e05cc8b8784a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.252811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.253430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.253711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.254008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.255275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:31.255581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.257290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.276777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1028/2061] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:31.278619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.294808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586d14a-34b1-4844-b32d-573625dc3477 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.301356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:31.310690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2159330a7d4547929d82b528df3e6b19', 'request_id': 'req-c063e086-54e2-4a92-9651-01e493c90f93'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.320806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2159330a7d4547929d82b528df3e6b19', 'request_id': 'req-c063e086-54e2-4a92-9651-01e493c90f93'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.328586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2159330a7d4547929d82b528df3e6b19', 'request_id': 'req-c063e086-54e2-4a92-9651-01e493c90f93'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.343482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] The token cache is being invalidate because project 2159330a7d4547929d82b528df3e6b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:31.344356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.345196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.346988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1034/2062] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:31 2022] DELETE /identity/v3/projects/2159330a7d4547929d82b528df3e6b19 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:31.351454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.356182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:31.393982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.394734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.395108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.395501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.396119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.398257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.431538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1029/2063] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:31 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:31.436249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.438128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:31.446083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063e086-54e2-4a92-9651-01e493c90f93 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.516558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.517282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.517579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.517829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] PATH_INFO: `/v3/users/4436d4c869ed4e6687875193789552ec` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.519027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4436d4c869ed4e6687875193789552ec)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.520976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.522799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ab2f4-5fb4-444d-9c2c-c5a8d3574b96 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.528050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.528833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.529297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.529649 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.530379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.532350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.561715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:31.573297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1030/2064] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:31 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:31.577371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.586206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4436d4c869ed4e6687875193789552ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.589759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4151e95-1721-4c81-95b2-2558ed9da12f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.595793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.596543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.596650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4436d4c869ed4e6687875193789552ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.596878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.597152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.598065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.599508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.615171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1031/2065] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:31.616156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4436d4c869ed4e6687875193789552ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.616603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.626271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4436d4c869ed4e6687875193789552ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.627277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4436d4c869ed4e6687875193789552ec'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:31.628931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:31.630123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0d8bc-bfd2-4bb3-8e4b-cb58fdc6210d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.635855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e010afb-04e4-4982-a954-9d8103800a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.636485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e010afb-04e4-4982-a954-9d8103800a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.636777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e010afb-04e4-4982-a954-9d8103800a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.637062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e010afb-04e4-4982-a954-9d8103800a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.637751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e010afb-04e4-4982-a954-9d8103800a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.640367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e010afb-04e4-4982-a954-9d8103800a09 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.651415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1035/2066] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:31 2022] DELETE /identity/v3/users/4436d4c869ed4e6687875193789552ec => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:31.674786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1032/2067] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:31.678015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:31.681239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:31.782605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91bf98-6ffd-4a3a-9145-0fba40e29d66 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:31.787767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:31.788783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.789169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.789523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] PATH_INFO: `/v3/projects/470696736c774d17a979b41751709eec/users/3d8acdc7d9e040939f77c6e1f966d560/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.822263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470696736c774d17a979b41751709eec, user_id=3d8acdc7d9e040939f77c6e1f966d560, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:31.828894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:31.858629 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1036/2068] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:31 2022] PUT /identity/v3/projects/470696736c774d17a979b41751709eec/users/3d8acdc7d9e040939f77c6e1f966d560/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:31.871772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:31.872053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:31.872322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:31.978186 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] MFA Rules not processed for user `3d8acdc7d9e040939f77c6e1f966d560`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:31.979061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.108300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1033/2069] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:32.112672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.194442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.203829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.221339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261169a2-3f49-42a6-8000-fabd6f5c4784 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.227148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b54668-7228-4825-8b1d-3be75aee7763 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.228339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.229218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.229566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.229922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] PATH_INFO: `/v3/projects/359e0e13503142fcb4ad492ca4c675b9` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.231625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=359e0e13503142fcb4ad492ca4c675b9)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.233436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.242364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.243431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.243431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.243646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.244173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.246232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.287072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1037/2070] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/projects => generated 387 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:32.291002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.307941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '359e0e13503142fcb4ad492ca4c675b9', 'request_id': 'req-b62a2ee2-012b-469d-96e5-7496763d1e32'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:32.319292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '359e0e13503142fcb4ad492ca4c675b9', 'request_id': 'req-b62a2ee2-012b-469d-96e5-7496763d1e32'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:32.327880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '359e0e13503142fcb4ad492ca4c675b9', 'request_id': 'req-b62a2ee2-012b-469d-96e5-7496763d1e32'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:32.339688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] The token cache is being invalidate because project 359e0e13503142fcb4ad492ca4c675b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:32.340303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:32.341193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-b62a2ee2-012b-469d-96e5-7496763d1e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:32.343444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1034/2071] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:32 2022] DELETE /identity/v3/projects/359e0e13503142fcb4ad492ca4c675b9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:32.347860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4717-a24c-4f6a-a041-de414ac804e4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.348152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4717-a24c-4f6a-a041-de414ac804e4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.348428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4717-a24c-4f6a-a041-de414ac804e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.362017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964bdb-3d0f-4fc0-b58e-7c890b01e797 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.369088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a4aa4717-a24c-4f6a-a041-de414ac804e4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:32.370280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a4aa4717-a24c-4f6a-a041-de414ac804e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.370925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:32.422960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:32.427969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.428575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.428903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.429170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.429807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.431877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.441382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1035/2072] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:32.451091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a7ba-58a7-41a9-b0fb-34c307fbb7d4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.451520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a7ba-58a7-41a9-b0fb-34c307fbb7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.451773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a7ba-58a7-41a9-b0fb-34c307fbb7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.459060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1038/2073] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:32.463762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.465392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.474911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-bbb4a7ba-58a7-41a9-b0fb-34c307fbb7d4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:32.476512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bbb4a7ba-58a7-41a9-b0fb-34c307fbb7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.478014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e699f-d5fe-4b12-a1c8-071da44c5cbb admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.483607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.484291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.484642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.485074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.485935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.486533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1036/2074] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:32.487893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.490902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4a7ba-58a7-41a9-b0fb-34c307fbb7d4 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.503664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1039/2075] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:32.503855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4a7ba-58a7-41a9-b0fb-34c307fbb7d4 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.504667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.511481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.512320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.512518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.512888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.514076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:32.514441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.515584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c731-202e-42cb-b2e1-1a7e46eeee72 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.516242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.521154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.521810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.522092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.522366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] PATH_INFO: `/v3/projects/4e679bab76664dd0b08b0a05f53e3483/users/a45bb79b7f264c178281a3a3053cfdc1/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.531170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1037/2076] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:00:32.535957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.541861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e679bab76664dd0b08b0a05f53e3483, user_id=a45bb79b7f264c178281a3a3053cfdc1, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.545366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.546566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0d679-b8de-4098-84d6-e5b56cbf46c2 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.551558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.552156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.552445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.552737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.553398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.555444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.559573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1040/2077] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:32 2022] PUT /identity/v3/projects/4e679bab76664dd0b08b0a05f53e3483/users/a45bb79b7f264c178281a3a3053cfdc1/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:32.564221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.588091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1038/2078] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:32.589099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.630293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d1117-fa62-4cb0-833e-507e1185b44f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.656850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82e80f-d0d5-4caa-b7ff-61bbd03e09ab admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.661708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.662311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.662588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.662859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.663491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.665590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.693858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1039/2079] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:32.694057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.694477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.694748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.694941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.695341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.695984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.697625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.705202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9d87c-572d-4ab4-8adb-18ae3e1233ae admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.710915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1041/2080] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:32.711040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.711040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.711040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.711274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.711803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.711899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.713610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.723641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2b3b7-7f9d-460a-b84e-02f10d194b70 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.724694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1040/2081] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:32.728544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.729310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d2f09-e0c7-4ec2-baca-015f5900560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.730050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d2f09-e0c7-4ec2-baca-015f5900560e admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.730321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d2f09-e0c7-4ec2-baca-015f5900560e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.730587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d2f09-e0c7-4ec2-baca-015f5900560e admin admin] PATH_INFO: `/v3/projects/4e679bab76664dd0b08b0a05f53e3483/users/a45bb79b7f264c178281a3a3053cfdc1/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.733315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d2f09-e0c7-4ec2-baca-015f5900560e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e679bab76664dd0b08b0a05f53e3483, user_id=a45bb79b7f264c178281a3a3053cfdc1, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.736155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d2f09-e0c7-4ec2-baca-015f5900560e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.739200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f76a30-c69d-4825-aaca-2a7b2b660510 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.744085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:32.750632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.751352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.751629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.751898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.752549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.754410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.756184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1042/2082] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:32 2022] PUT /identity/v3/projects/4e679bab76664dd0b08b0a05f53e3483/users/a45bb79b7f264c178281a3a3053cfdc1/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:32.768674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1041/2083] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:32.768864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.768864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.768864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.769698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.780511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924cfaa-8dd7-49df-8bc0-dae016af120e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.799403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] MFA Rules not processed for user `a45bb79b7f264c178281a3a3053cfdc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:32.800607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.842959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadef38-4983-4093-a9f3-d029400f13b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:32.844038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadef38-4983-4093-a9f3-d029400f13b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.844323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadef38-4983-4093-a9f3-d029400f13b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.844592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadef38-4983-4093-a9f3-d029400f13b3 admin admin] PATH_INFO: `/v3/projects/4aa21ed0f35247f98cc92b702b1dc42b/users/79014bca69ee4c78a163e5a1cc467114/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.859782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadef38-4983-4093-a9f3-d029400f13b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aa21ed0f35247f98cc92b702b1dc42b, user_id=79014bca69ee4c78a163e5a1cc467114, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:32.862511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadef38-4983-4093-a9f3-d029400f13b3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:32.870799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:32.874153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1043/2084] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:32.881841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1042/2085] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:32 2022] PUT /identity/v3/projects/4aa21ed0f35247f98cc92b702b1dc42b/users/79014bca69ee4c78a163e5a1cc467114/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:32.892817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:32.893353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff172bd-7c90-4310-8bad-65e1636ca546 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:32.893610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff172bd-7c90-4310-8bad-65e1636ca546 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:32.893882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff172bd-7c90-4310-8bad-65e1636ca546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:32.894294 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.921913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1ff172bd-7c90-4310-8bad-65e1636ca546 None None] MFA Rules not processed for user `79014bca69ee4c78a163e5a1cc467114`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:32.923101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1ff172bd-7c90-4310-8bad-65e1636ca546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:32.955426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5882d52-29a1-4501-96dc-adcb5b741e1d None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:32.995202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1043/2086] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:33.000403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff172bd-7c90-4310-8bad-65e1636ca546 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.017678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.018563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.018972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.019295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.085818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:33.099274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff172bd-7c90-4310-8bad-65e1636ca546 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.103240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.104686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.107725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.108342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.108618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.108922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.109563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.117861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1044/2087] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:33.118133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.173997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1044/2088] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:32 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:33.178083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.198406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ba124-da38-4bb6-9200-c496fbc5f2c7 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.205311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.270175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.270869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.271228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.271559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.272279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.274578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.298683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1d056-731d-400b-a7c5-d8ae1bffb332 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.299254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1045/2089] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:33 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:33.305868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.306727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.307075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.307423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] PATH_INFO: `/v3/users/3d8acdc7d9e040939f77c6e1f966d560` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.308465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.308880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8acdc7d9e040939f77c6e1f966d560)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.311669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.320405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98544aa-2fc3-47ce-bde2-626cc1e35be4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.325301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.325932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.326204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.326468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.327015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:33.327149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.328684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.339264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1046/2090] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:33.344302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.354781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5037c4-e886-4fd5-98e4-34eb507ccfe6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.362137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8acdc7d9e040939f77c6e1f966d560'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:33.362908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8acdc7d9e040939f77c6e1f966d560'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:33.364351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.367709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.367999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.368273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.368994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.370573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.371893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8acdc7d9e040939f77c6e1f966d560'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:33.382019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1047/2091] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:33.386212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.386450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8acdc7d9e040939f77c6e1f966d560'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:33.388114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:33.394965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7a80c377-e5eb-47e0-8de2-32b6392835dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8acdc7d9e040939f77c6e1f966d560'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:33.401317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df74457-9ab9-44f5-a77e-f4bff7e3326a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.402240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1045/2092] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:33 2022] DELETE /identity/v3/users/3d8acdc7d9e040939f77c6e1f966d560 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:33.407068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437a2a-f7f1-42b1-b196-4bdc13bb2964 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.407274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437a2a-f7f1-42b1-b196-4bdc13bb2964 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.407548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab437a2a-f7f1-42b1-b196-4bdc13bb2964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.429721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ab437a2a-f7f1-42b1-b196-4bdc13bb2964 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:33.430917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ab437a2a-f7f1-42b1-b196-4bdc13bb2964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:33.458957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.459621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.459893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.460163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] PATH_INFO: `/v3/projects/a79a6daa28ba441f882e04c4e06efe12/users/30e61415318e4529b901e8e94a420ce8/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.475830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79a6daa28ba441f882e04c4e06efe12, user_id=30e61415318e4529b901e8e94a420ce8, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.478537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.492538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1048/2093] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:33 2022] PUT /identity/v3/projects/a79a6daa28ba441f882e04c4e06efe12/users/30e61415318e4529b901e8e94a420ce8/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:33.496873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1046/2094] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:33.503277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052667b2-7eba-456d-87e1-107b9eb7bf19 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.503566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052667b2-7eba-456d-87e1-107b9eb7bf19 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.503842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052667b2-7eba-456d-87e1-107b9eb7bf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.507025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.508681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:33.536385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-052667b2-7eba-456d-87e1-107b9eb7bf19 None None] MFA Rules not processed for user `30e61415318e4529b901e8e94a420ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:33.537642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-052667b2-7eba-456d-87e1-107b9eb7bf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:33.560610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:33.579639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e81ee99-2fa4-4944-b3e0-2133a58d55fa admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.589415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.590112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.590446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.590820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.618133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1047/2095] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:33.622502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052667b2-7eba-456d-87e1-107b9eb7bf19 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.651766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:33.664707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.666266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.677096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1049/2096] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:33.703504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052667b2-7eba-456d-87e1-107b9eb7bf19 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.708669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.709391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.709668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.709939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.710581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.712505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.746193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1048/2097] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:33 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:33.747524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.757597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f7b29-0f04-4aa9-8061-dbfff37a2523 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.827901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.828515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.828822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.829217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.829862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.832742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.861056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1050/2098] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:33 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:33.865332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.876297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce895b78-a686-4024-abaa-a9e3f53da45a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.881273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.881985 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.882294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.882568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.883220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.884939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.899530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1049/2099] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:33.900783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.912060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777795d-2e89-47bb-8ace-0f6b463d3ac9 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:33.916896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:33.917541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:33.917819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:33.918088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] PATH_INFO: `/v3/projects/bbd31cf8b49d43aaac31930f31280778/users/71bc8be3d69244409500f7a1467a4536/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:33.938385 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd31cf8b49d43aaac31930f31280778, user_id=71bc8be3d69244409500f7a1467a4536, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:33.941447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:33.955308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.957165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:33.959410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1051/2100] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:33 2022] PUT /identity/v3/projects/bbd31cf8b49d43aaac31930f31280778/users/71bc8be3d69244409500f7a1467a4536/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:33.970862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:33.976217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc3820-81af-4566-9248-59282f859d2c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:34.038171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:34.045555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1cedb-b7c6-4972-bb6c-003fc9607511 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:34.052142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:34.052695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:34.053041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.053324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.053601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.053993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.054280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:34.054597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.055122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] PATH_INFO: `/v3/projects/470696736c774d17a979b41751709eec` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.055916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:34.056598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=470696736c774d17a979b41751709eec)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:34.059398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:34.068165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1052/2101] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:34.072340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:34.083793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cd28d-6d65-4a54-90c6-a6f08b3d5818 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:34.101411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327de5-9514-4f2a-aa20-44a2872e0b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:34.102084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327de5-9514-4f2a-aa20-44a2872e0b23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.102366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327de5-9514-4f2a-aa20-44a2872e0b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.102637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327de5-9514-4f2a-aa20-44a2872e0b23 admin admin] PATH_INFO: `/v3/projects/bbd31cf8b49d43aaac31930f31280778/users/71bc8be3d69244409500f7a1467a4536/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.104564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327de5-9514-4f2a-aa20-44a2872e0b23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd31cf8b49d43aaac31930f31280778, user_id=71bc8be3d69244409500f7a1467a4536, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:34.107655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327de5-9514-4f2a-aa20-44a2872e0b23 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:34.113997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470696736c774d17a979b41751709eec', 'request_id': 'req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.118481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1053/2102] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:34 2022] PUT /identity/v3/projects/bbd31cf8b49d43aaac31930f31280778/users/71bc8be3d69244409500f7a1467a4536/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:34.121853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470696736c774d17a979b41751709eec', 'request_id': 'req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.128953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470696736c774d17a979b41751709eec', 'request_id': 'req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.130516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.130807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.131083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.146398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] The token cache is being invalidate because project 470696736c774d17a979b41751709eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:34.146769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.147520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.149250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1050/2103] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:33 2022] DELETE /identity/v3/projects/470696736c774d17a979b41751709eec => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:34.153285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:34.155582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:34.166978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] MFA Rules not processed for user `71bc8be3d69244409500f7a1467a4536`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:34.168187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:34.189975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:34.231772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac628968-378a-4eb8-a1b9-fde1df1cc1e2 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:34.256131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1054/2104] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:34.268791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:34.270198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:34.307412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:34.308053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.308328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.308598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] PATH_INFO: `/v3/users/a45bb79b7f264c178281a3a3053cfdc1` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.310522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45bb79b7f264c178281a3a3053cfdc1)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:34.312612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:34.387155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2e38e-392d-4b41-be10-b77faa8b5c9f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:34.391479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:34.392819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.392819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.393231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.394604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45bb79b7f264c178281a3a3053cfdc1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.395352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45bb79b7f264c178281a3a3053cfdc1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.406644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:34.407201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45bb79b7f264c178281a3a3053cfdc1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.408509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:34.421801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1055/2105] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:34.425756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45bb79b7f264c178281a3a3053cfdc1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.433327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45bb79b7f264c178281a3a3053cfdc1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.452814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1051/2106] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:34 2022] DELETE /identity/v3/users/a45bb79b7f264c178281a3a3053cfdc1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:34.475334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:34.477353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:34.505178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28267483-a423-4faa-80fa-1941b742ed16 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:34.563287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34458fc-def4-4c65-92c2-734599509f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:34.563905 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34458fc-def4-4c65-92c2-734599509f48 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.564184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34458fc-def4-4c65-92c2-734599509f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.564458 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34458fc-def4-4c65-92c2-734599509f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.566411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c34458fc-def4-4c65-92c2-734599509f48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:34.593605 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:34.643435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34458fc-def4-4c65-92c2-734599509f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:34.645161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34458fc-def4-4c65-92c2-734599509f48 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:34.654752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1056/2107] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:34.764555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:34.766227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:34.829062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:34.843342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84ff20-e80b-4e8f-987c-6cc63104a745 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:34.849577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:34.850278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.850852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.851488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] PATH_INFO: `/v3/projects/4e679bab76664dd0b08b0a05f53e3483` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.853457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e679bab76664dd0b08b0a05f53e3483)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:34.855927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:34.911531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e679bab76664dd0b08b0a05f53e3483', 'request_id': 'req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.920346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e679bab76664dd0b08b0a05f53e3483', 'request_id': 'req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.926545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e679bab76664dd0b08b0a05f53e3483', 'request_id': 'req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.941616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] The token cache is being invalidate because project 4e679bab76664dd0b08b0a05f53e3483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:34.942078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.942783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-4e0cbe62-0a9e-4bbb-86cc-6d94c7fdb825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:34.945172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1052/2108] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:34 2022] DELETE /identity/v3/projects/4e679bab76664dd0b08b0a05f53e3483 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:34.949916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416349e-f426-4b19-b347-b9188fd288bd None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:34.950198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416349e-f426-4b19-b347-b9188fd288bd None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:34.950514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416349e-f426-4b19-b347-b9188fd288bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:34.971398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a416349e-f426-4b19-b347-b9188fd288bd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:34.972799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a416349e-f426-4b19-b347-b9188fd288bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:35.062251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1057/2109] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:35.070204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a2a9b-b54a-43a4-b34a-7094c8b640a1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.070722 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a2a9b-b54a-43a4-b34a-7094c8b640a1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.070722 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a2a9b-b54a-43a4-b34a-7094c8b640a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.091980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-f83a2a9b-b54a-43a4-b34a-7094c8b640a1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:35.093510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f83a2a9b-b54a-43a4-b34a-7094c8b640a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:35.103270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1053/2110] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:35.107499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416349e-f426-4b19-b347-b9188fd288bd None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.117447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416349e-f426-4b19-b347-b9188fd288bd None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.123748 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.124389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.124681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.125011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.125584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:35.125887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.127486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.140146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1058/2111] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:35.144463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a2a9b-b54a-43a4-b34a-7094c8b640a1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.154458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a2a9b-b54a-43a4-b34a-7094c8b640a1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.159130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.159727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.160007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.160278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.161046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.162986 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.190031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1054/2112] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:35.194246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.207446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc54e4-ebcb-45fd-8e8d-a1ece4ee1e4c admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.265656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.266252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.266524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.266789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.267418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.269363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.290183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1059/2113] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:35.294469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.304675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba4c5-735e-49a7-84c7-baa0f082f064 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.309421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.310516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.310792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.311135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.311727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.313388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.323817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1055/2114] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:35.328330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.338022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ee9f-1aa4-47ad-a150-90813f6710ad admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.342783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ae40f-1da5-4952-85de-f1f9ce5d1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.343365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ae40f-1da5-4952-85de-f1f9ce5d1436 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.343639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ae40f-1da5-4952-85de-f1f9ce5d1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.343901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ae40f-1da5-4952-85de-f1f9ce5d1436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.344530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ae40f-1da5-4952-85de-f1f9ce5d1436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.346191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ae40f-1da5-4952-85de-f1f9ce5d1436 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.356040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1060/2115] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:35.360165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.370086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d033be7-c51c-43a0-bdfb-37a9bea3dcaf admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.375619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43837e43-2c2a-4ae2-a727-b2c1c1c50a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.376269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43837e43-2c2a-4ae2-a727-b2c1c1c50a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.376904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43837e43-2c2a-4ae2-a727-b2c1c1c50a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.377221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43837e43-2c2a-4ae2-a727-b2c1c1c50a2a admin admin] PATH_INFO: `/v3/projects/58bab1a9aecb48a4a7b18d6f7f88f74c/users/088796cfb8bc4d0693a1c16f63399320/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.394499 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43837e43-2c2a-4ae2-a727-b2c1c1c50a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58bab1a9aecb48a4a7b18d6f7f88f74c, user_id=088796cfb8bc4d0693a1c16f63399320, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.397132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43837e43-2c2a-4ae2-a727-b2c1c1c50a2a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.412131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1056/2116] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:35 2022] PUT /identity/v3/projects/58bab1a9aecb48a4a7b18d6f7f88f74c/users/088796cfb8bc4d0693a1c16f63399320/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:35.423426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ebb1-bc1c-469e-bffb-0e578cbff4df None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.424009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ebb1-bc1c-469e-bffb-0e578cbff4df None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.424291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3ebb1-bc1c-469e-bffb-0e578cbff4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.453700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-70d3ebb1-bc1c-469e-bffb-0e578cbff4df None None] MFA Rules not processed for user `088796cfb8bc4d0693a1c16f63399320`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:35.454933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-70d3ebb1-bc1c-469e-bffb-0e578cbff4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:35.470756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:35.556183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef243-230e-465b-a462-3cb505cecd77 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.556689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef243-230e-465b-a462-3cb505cecd77 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.557198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef243-230e-465b-a462-3cb505cecd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.557328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1061/2117] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:35.562361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3ebb1-bc1c-469e-bffb-0e578cbff4df None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.579491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a03ef243-230e-465b-a462-3cb505cecd77 None None] MFA Rules not processed for user `eeb6f39c8c2b4a2f9ae77a20a20b655c`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:35.580691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a03ef243-230e-465b-a462-3cb505cecd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:35.651782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3ebb1-bc1c-469e-bffb-0e578cbff4df None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.659598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1057/2118] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:35.661693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.662591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.663049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.663904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.664902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.667115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.699128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1062/2119] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:35.703460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef243-230e-465b-a462-3cb505cecd77 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.714931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef243-230e-465b-a462-3cb505cecd77 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.742648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:35.780641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.781276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.781556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.781827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.782471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.784431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.806391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1058/2120] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:35.811353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.823609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7894d1c-54d2-4813-b53b-5edc1f7ae4bd admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.829420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.830048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.830349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.830623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.831278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.832965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.844415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1063/2121] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:35.848325 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.858708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f7caa-6a2c-4779-acb9-3d3246468a92 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.863546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89561a0b-01bf-482c-9173-78ab5e75b05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.864146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89561a0b-01bf-482c-9173-78ab5e75b05d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.864420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89561a0b-01bf-482c-9173-78ab5e75b05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.864685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89561a0b-01bf-482c-9173-78ab5e75b05d admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.865354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89561a0b-01bf-482c-9173-78ab5e75b05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.866960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89561a0b-01bf-482c-9173-78ab5e75b05d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.880758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1059/2122] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:35.883096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:35.894816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef86b99-a472-49ef-b68d-e6cf771d4410 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:35.900133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:35.900817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.901102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.901379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] PATH_INFO: `/v3/projects/508b352baaff4fe0a055023d6fc8a2d6/users/b960f3ec76044d6a96b70baa1da80463/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.920760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=508b352baaff4fe0a055023d6fc8a2d6, user_id=b960f3ec76044d6a96b70baa1da80463, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:35.921209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:35.936114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1064/2123] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:35 2022] PUT /identity/v3/projects/508b352baaff4fe0a055023d6fc8a2d6/users/b960f3ec76044d6a96b70baa1da80463/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:35.947398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a596-f1c9-41d1-a0ee-d26e7bb18f00 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:35.947680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a596-f1c9-41d1-a0ee-d26e7bb18f00 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:35.947961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41a596-f1c9-41d1-a0ee-d26e7bb18f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:35.979782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-4b41a596-f1c9-41d1-a0ee-d26e7bb18f00 None None] MFA Rules not processed for user `b960f3ec76044d6a96b70baa1da80463`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:35.981076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-4b41a596-f1c9-41d1-a0ee-d26e7bb18f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:36.050877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1060/2124] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:36.055834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:36.135046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d84aa-d686-41e8-9412-c38992d396d8 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:36.140529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:36.141416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.141766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.142706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.143452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:36.145736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:36.161553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:36.180749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1065/2125] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:36 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:36.187162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a596-f1c9-41d1-a0ee-d26e7bb18f00 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:36.199381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41a596-f1c9-41d1-a0ee-d26e7bb18f00 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:36.302357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:36.302990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.304328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.304442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.315339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:36.317849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:36.350340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1061/2126] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:36 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:36.354336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:36.366297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99440401-a0fe-4913-93a7-e1a6a5483d0b admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:36.374985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:36.375606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.375934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.376205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.377132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:36.378815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:36.394281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1066/2127] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:36.402308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:36.423800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fb7ce-b835-4b98-a956-0a8d5d4a870a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:36.428913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:36.429610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.430559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.430838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] PATH_INFO: `/v3/projects/95912f4cf6964f13a6ba636bf133ffda/users/a961968a9bd1493b823dd5296596f640/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.460831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] RBAC: Authorizing `identity:create_grant(project_id=95912f4cf6964f13a6ba636bf133ffda, user_id=a961968a9bd1493b823dd5296596f640, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:36.466942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:36.485934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:36.500788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1062/2128] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:36 2022] PUT /identity/v3/projects/95912f4cf6964f13a6ba636bf133ffda/users/a961968a9bd1493b823dd5296596f640/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:36.503353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:36.514824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c9a25-737c-4d90-b6d8-d09fdb915a10 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:36.611651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76d1a0-c640-48ba-ae24-c07366d62e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:36.613021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d1a0-c640-48ba-ae24-c07366d62e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.613308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d1a0-c640-48ba-ae24-c07366d62e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.613676 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76d1a0-c640-48ba-ae24-c07366d62e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.614470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76d1a0-c640-48ba-ae24-c07366d62e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:36.616529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76d1a0-c640-48ba-ae24-c07366d62e3b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:36.631669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1067/2129] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:36.636739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:36.651815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e7692-fa97-4fe1-ae95-11a36690147d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:36.665070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:36.667581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.667886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.668634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] PATH_INFO: `/v3/projects/95912f4cf6964f13a6ba636bf133ffda/users/a961968a9bd1493b823dd5296596f640/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.684332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] RBAC: Authorizing `identity:create_grant(project_id=95912f4cf6964f13a6ba636bf133ffda, user_id=a961968a9bd1493b823dd5296596f640, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:36.689414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:36.707715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1063/2130] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:36 2022] PUT /identity/v3/projects/95912f4cf6964f13a6ba636bf133ffda/users/a961968a9bd1493b823dd5296596f640/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:36.719494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb64e43-620c-47a7-a60c-aa3c03f571b2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.720035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb64e43-620c-47a7-a60c-aa3c03f571b2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.720554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb64e43-620c-47a7-a60c-aa3c03f571b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.771329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ecb64e43-620c-47a7-a60c-aa3c03f571b2 None None] MFA Rules not processed for user `a961968a9bd1493b823dd5296596f640`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:36.772705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ecb64e43-620c-47a7-a60c-aa3c03f571b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:36.881934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1068/2131] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:36.896673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:36.898552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:36.969335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb93428-44f4-4128-99a3-b8ba60a6c97e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:36.973276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:36.973911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:36.974191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:36.974481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:36.984523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:36.986594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:36.997510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1064/2132] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:37.558584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb64e43-620c-47a7-a60c-aa3c03f571b2 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:37.569091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb64e43-620c-47a7-a60c-aa3c03f571b2 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:37.572796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8787c-2a78-4741-b1a9-2963530f6675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:37.573423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8787c-2a78-4741-b1a9-2963530f6675 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:37.573702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8787c-2a78-4741-b1a9-2963530f6675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:37.573976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8787c-2a78-4741-b1a9-2963530f6675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:37.634771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8787c-2a78-4741-b1a9-2963530f6675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:37.636223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8787c-2a78-4741-b1a9-2963530f6675 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:37.641518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:37.646416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1069/2133] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:38.065730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:38.067396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:38.168059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547533c1-ddc0-4071-a5da-a35014e56adf service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:38.172996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:38.173723 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:38.174004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:38.174284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] PATH_INFO: `/v3/users/79014bca69ee4c78a163e5a1cc467114` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:38.175474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79014bca69ee4c78a163e5a1cc467114)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:38.177469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:38.233102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79014bca69ee4c78a163e5a1cc467114'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.233747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79014bca69ee4c78a163e5a1cc467114'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.241159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79014bca69ee4c78a163e5a1cc467114'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.254567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79014bca69ee4c78a163e5a1cc467114'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.261720 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79014bca69ee4c78a163e5a1cc467114'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.268673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1065/2134] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:38 2022] DELETE /identity/v3/users/79014bca69ee4c78a163e5a1cc467114 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:38.273299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:38.273651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:38.273992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:38.293183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:38.295041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:38.367224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1070/2135] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:38.378522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:38.380337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:38.391797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:38.446019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedceaf-184f-4885-8b04-4fece3322dc2 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:38.449667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:38.450302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:38.451141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:38.451141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:38.461251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:38.462721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:38.473923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1066/2136] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:38.766151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:38.767795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:38.781295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac680d9-8a76-4b29-9bff-56931a8ab2f3 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:38.786298 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:38.787089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:38.787386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:38.787630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] PATH_INFO: `/v3/projects/4aa21ed0f35247f98cc92b702b1dc42b` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:38.788807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aa21ed0f35247f98cc92b702b1dc42b)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:38.790721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:38.850305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aa21ed0f35247f98cc92b702b1dc42b', 'request_id': 'req-5aff766b-7313-4636-ba89-cadae4107422'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.859358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aa21ed0f35247f98cc92b702b1dc42b', 'request_id': 'req-5aff766b-7313-4636-ba89-cadae4107422'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.868202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aa21ed0f35247f98cc92b702b1dc42b', 'request_id': 'req-5aff766b-7313-4636-ba89-cadae4107422'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.884159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] The token cache is being invalidate because project 4aa21ed0f35247f98cc92b702b1dc42b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:38.884453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.885133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:38.886890 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1071/2137] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:38 2022] DELETE /identity/v3/projects/4aa21ed0f35247f98cc92b702b1dc42b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:38.891920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:38.902122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f8305-3583-4876-b59d-c25196674eec service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:38.963385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:38.964009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:38.964284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:38.964552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] PATH_INFO: `/v3/users/30e61415318e4529b901e8e94a420ce8` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:38.965707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e61415318e4529b901e8e94a420ce8)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:38.967957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:39.029856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e61415318e4529b901e8e94a420ce8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:39.031365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e61415318e4529b901e8e94a420ce8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:39.040405 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e61415318e4529b901e8e94a420ce8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:39.057105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e61415318e4529b901e8e94a420ce8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:39.066180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e61415318e4529b901e8e94a420ce8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:39.078217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1067/2138] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:38 2022] DELETE /identity/v3/users/30e61415318e4529b901e8e94a420ce8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:39.944247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:39.945694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:39.959169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff766b-7313-4636-ba89-cadae4107422 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:40.038556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:40.039189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:40.039463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:40.039745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] PATH_INFO: `/v3/projects/a79a6daa28ba441f882e04c4e06efe12` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:40.040924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a79a6daa28ba441f882e04c4e06efe12)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:40.042832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:40.078558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:40.102071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79a6daa28ba441f882e04c4e06efe12', 'request_id': 'req-6645ab59-0112-4a9f-9619-91665a54d925'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.110123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79a6daa28ba441f882e04c4e06efe12', 'request_id': 'req-6645ab59-0112-4a9f-9619-91665a54d925'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.116546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79a6daa28ba441f882e04c4e06efe12', 'request_id': 'req-6645ab59-0112-4a9f-9619-91665a54d925'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.135118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] The token cache is being invalidate because project a79a6daa28ba441f882e04c4e06efe12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:40.135619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.136382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.141230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1072/2139] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:39 2022] DELETE /identity/v3/projects/a79a6daa28ba441f882e04c4e06efe12 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:40.145204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:40.146877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:40.160654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d968cdd-5953-4a6e-ab59-e5cef790ff79 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:40.191109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:40.234127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:40.234776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:40.235060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:40.235337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] PATH_INFO: `/v3/users/71bc8be3d69244409500f7a1467a4536` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:40.236967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bc8be3d69244409500f7a1467a4536)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:40.238932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:40.311973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bc8be3d69244409500f7a1467a4536'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.312658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bc8be3d69244409500f7a1467a4536'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.323084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bc8be3d69244409500f7a1467a4536'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.344210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bc8be3d69244409500f7a1467a4536'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.355399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-442cfda8-d9dc-4aa8-8348-70d851f664bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bc8be3d69244409500f7a1467a4536'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.364090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1068/2140] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:40 2022] DELETE /identity/v3/users/71bc8be3d69244409500f7a1467a4536 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:40.689534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:40.691297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:40.705410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645ab59-0112-4a9f-9619-91665a54d925 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:40.790402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:40.791037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:40.791316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:40.791587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] PATH_INFO: `/v3/projects/bbd31cf8b49d43aaac31930f31280778` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:40.792706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd31cf8b49d43aaac31930f31280778)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:40.795234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:40.849557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd31cf8b49d43aaac31930f31280778', 'request_id': 'req-98720637-ee25-4af1-94d3-fabce7851f16'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.858448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd31cf8b49d43aaac31930f31280778', 'request_id': 'req-98720637-ee25-4af1-94d3-fabce7851f16'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.864403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd31cf8b49d43aaac31930f31280778', 'request_id': 'req-98720637-ee25-4af1-94d3-fabce7851f16'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.875772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] The token cache is being invalidate because project bbd31cf8b49d43aaac31930f31280778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:40.876054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.876704 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-98720637-ee25-4af1-94d3-fabce7851f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:40.878275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1073/2141] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:40 2022] DELETE /identity/v3/projects/bbd31cf8b49d43aaac31930f31280778 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:40.884347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01df26a-5660-4dad-9c9a-83ccc48c4bca None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:40.884871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01df26a-5660-4dad-9c9a-83ccc48c4bca None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:40.885152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01df26a-5660-4dad-9c9a-83ccc48c4bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:40.908012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e01df26a-5660-4dad-9c9a-83ccc48c4bca None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:40.909270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e01df26a-5660-4dad-9c9a-83ccc48c4bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:40.974615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1069/2142] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:40.982950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e2e02-830f-4117-a876-ebdb428e622f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:40.983489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e2e02-830f-4117-a876-ebdb428e622f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:40.983489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e2e02-830f-4117-a876-ebdb428e622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.003870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-806e2e02-830f-4117-a876-ebdb428e622f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:41.005216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-806e2e02-830f-4117-a876-ebdb428e622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.013917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1074/2143] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:41.018210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01df26a-5660-4dad-9c9a-83ccc48c4bca None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.027944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01df26a-5660-4dad-9c9a-83ccc48c4bca None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.033793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.034386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.034665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.034939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.035583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:41.035885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.037540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.050091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1070/2144] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:41.054765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e2e02-830f-4117-a876-ebdb428e622f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.065737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e2e02-830f-4117-a876-ebdb428e622f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.070431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.071023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.071300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.071569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.072192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.074117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.103259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1075/2145] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:41 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:41.107755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.118565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03e4d4-b415-4dd8-b442-721d744c8e50 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.152373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:41.182783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.183563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.183844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.184110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.185313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.187437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.210573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1071/2146] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:41 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:41.216058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.227357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c047-11ac-46f2-bddb-85ef16760e59 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.232021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.232619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.232933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.233205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.233853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.235575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.255263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1076/2147] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:41.256553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.268095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef15dfe-f39a-49ea-b71e-7e9ff66a5952 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.272694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980c9cb-b153-4e06-a0ee-c7772e3ab914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.273311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980c9cb-b153-4e06-a0ee-c7772e3ab914 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.273966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980c9cb-b153-4e06-a0ee-c7772e3ab914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.273966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980c9cb-b153-4e06-a0ee-c7772e3ab914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.274456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980c9cb-b153-4e06-a0ee-c7772e3ab914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.276475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980c9cb-b153-4e06-a0ee-c7772e3ab914 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.286941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1072/2148] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:41.291056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.300130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:41.304404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f046e74-d4f2-45e6-84b5-8f7a783b37e8 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.312479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.313189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.318884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.319162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] PATH_INFO: `/v3/projects/2b35b523c3b84871afc0ce5db7950145/users/50f0124ba65346fea4498894e516f7d7/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.339767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b35b523c3b84871afc0ce5db7950145, user_id=50f0124ba65346fea4498894e516f7d7, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.342944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.357456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1077/2149] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:41 2022] PUT /identity/v3/projects/2b35b523c3b84871afc0ce5db7950145/users/50f0124ba65346fea4498894e516f7d7/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:41.369783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.369994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.370276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.411145 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.411306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] MFA Rules not processed for user `50f0124ba65346fea4498894e516f7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:41.412443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.412872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.479035 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1073/2150] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:41.482315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dbecc-e552-4ec2-866a-7661d1f740a7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.487272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.487911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.488188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.488456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] PATH_INFO: `/v3/users/78f0f1cce6664c3894910230f27d5a02` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.489641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=78f0f1cce6664c3894910230f27d5a02)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.491275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.491587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.492704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.564394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21152cdb-7724-4ead-94ac-ee805072dafe None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.565581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78f0f1cce6664c3894910230f27d5a02'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.568809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.569427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.569833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.570112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.582482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.584385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.586690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78f0f1cce6664c3894910230f27d5a02'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.596102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1074/2151] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:41.613902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78f0f1cce6664c3894910230f27d5a02'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.618982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78f0f1cce6664c3894910230f27d5a02'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.619900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-113697c1-1603-48a9-b095-b27d0e130f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78f0f1cce6664c3894910230f27d5a02'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.620259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.627228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1078/2152] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:41 2022] DELETE /identity/v3/users/78f0f1cce6664c3894910230f27d5a02 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:41.631327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.631735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.632097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.632372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38a9b1-59b9-433b-8f77-70d01aed729e service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.651799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:41.653125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.687288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:41.704279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.705045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.705399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.705741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] PATH_INFO: `/v3/users/50f0124ba65346fea4498894e516f7d7` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.706975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f0124ba65346fea4498894e516f7d7)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.707688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:41.709717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.720680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1079/2153] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:41.729971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.732144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.768130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f0124ba65346fea4498894e516f7d7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.768947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f0124ba65346fea4498894e516f7d7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.778006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f0124ba65346fea4498894e516f7d7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.797713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f0124ba65346fea4498894e516f7d7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.806083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bbfa4-1d24-47c7-a3b9-cd807e84d7a8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:41.810870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-69f8e9ad-2bad-4397-8c4d-4d72633681d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f0124ba65346fea4498894e516f7d7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:41.811436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:41.812230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.812510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.812807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.815406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.819443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1075/2154] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:41 2022] DELETE /identity/v3/users/50f0124ba65346fea4498894e516f7d7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:41.823613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:41.823897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:41.824168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:41.837310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:41.838764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:41.850877 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:41.851662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1080/2155] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:41.852700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.957307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1076/2156] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:41.963835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:41.965317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:41.981591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f246b6-35ee-4105-9de6-899b06b7963d service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.096517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.097407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.097770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.098155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.116414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:42.118354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.150986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.152627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.178952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1081/2157] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:42.222583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe99c2-64b8-4862-a19c-664bdfa50a9b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.227995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad7b09-e550-49ef-9d78-db5295e52d72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.229105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad7b09-e550-49ef-9d78-db5295e52d72 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.229409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad7b09-e550-49ef-9d78-db5295e52d72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.229677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad7b09-e550-49ef-9d78-db5295e52d72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.232412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e7ad7b09-e550-49ef-9d78-db5295e52d72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.268273 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:42.276373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:42.279067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.302184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b7615-8beb-49ec-899a-593c681f6ad6 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.316791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.317540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.317820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.318090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] PATH_INFO: `/v3/projects/a36678835b4a4925a8278438f382c1c2` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.323529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36678835b4a4925a8278438f382c1c2)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.331904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.346745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad7b09-e550-49ef-9d78-db5295e52d72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.355228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad7b09-e550-49ef-9d78-db5295e52d72 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.378745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1077/2158] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:42.436783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36678835b4a4925a8278438f382c1c2', 'request_id': 'req-b88d0882-107e-454e-8f7e-3cc20cfd96e7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.445528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36678835b4a4925a8278438f382c1c2', 'request_id': 'req-b88d0882-107e-454e-8f7e-3cc20cfd96e7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.456214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36678835b4a4925a8278438f382c1c2', 'request_id': 'req-b88d0882-107e-454e-8f7e-3cc20cfd96e7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.468703 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] The token cache is being invalidate because project a36678835b4a4925a8278438f382c1c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:42.469015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.469767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b88d0882-107e-454e-8f7e-3cc20cfd96e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.471631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1082/2159] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:42 2022] DELETE /identity/v3/projects/a36678835b4a4925a8278438f382c1c2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:42.476330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.477259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.478029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.509210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:42.511232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.611032 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1078/2160] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:42.618694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b8675-e1c2-4855-a591-3b853a59ea21 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.619225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b8675-e1c2-4855-a591-3b853a59ea21 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.619493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b8675-e1c2-4855-a591-3b853a59ea21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.668326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ce2b8675-e1c2-4855-a591-3b853a59ea21 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:42.670656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ce2b8675-e1c2-4855-a591-3b853a59ea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.692309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:42.694305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.699169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1083/2161] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:42.704269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b8675-e1c2-4855-a591-3b853a59ea21 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:42.715358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41332a-2f2f-406e-93b0-ea203873f352 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.719186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b8675-e1c2-4855-a591-3b853a59ea21 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.722000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.722705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.723118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.723783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] PATH_INFO: `/v3/projects/2b35b523c3b84871afc0ce5db7950145` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.725647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b35b523c3b84871afc0ce5db7950145)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.727000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.727879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.728264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.728383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.728838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.729609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:42.730126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.733348 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.751819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1084/2162] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:42.756496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:42.769917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9b157-8949-47f8-9b42-2f390d49a898 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.776275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.776699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.778809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.779208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.779959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.782145 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.788169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:42.811166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b35b523c3b84871afc0ce5db7950145', 'request_id': 'req-083b863f-0820-4193-9578-6bec90d622a0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.821837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b35b523c3b84871afc0ce5db7950145', 'request_id': 'req-083b863f-0820-4193-9578-6bec90d622a0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.822282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1085/2163] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:42 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:42.827751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:42.832843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b35b523c3b84871afc0ce5db7950145', 'request_id': 'req-083b863f-0820-4193-9578-6bec90d622a0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.846968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] The token cache is being invalidate because project 2b35b523c3b84871afc0ce5db7950145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:42.847253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.847934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-083b863f-0820-4193-9578-6bec90d622a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:42.849763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1079/2164] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:42 2022] DELETE /identity/v3/projects/2b35b523c3b84871afc0ce5db7950145 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:42.855052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58d3b9-8665-44fd-971d-e9d8e3e637d5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.855679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58d3b9-8665-44fd-971d-e9d8e3e637d5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.855679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58d3b9-8665-44fd-971d-e9d8e3e637d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.879288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-3e58d3b9-8665-44fd-971d-e9d8e3e637d5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:42.881130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3e58d3b9-8665-44fd-971d-e9d8e3e637d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.907801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc0cbb-1a64-4997-95cd-4b3f64db1ff4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.913088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.913789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.914136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.914486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.915200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.917247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.929167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:42.948270 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1086/2165] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:42 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:42.952469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:42.954107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.957565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1080/2166] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:42.965235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21d703-d5c7-4561-865d-1d99ceab6f35 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.965524 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21d703-d5c7-4561-865d-1d99ceab6f35 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.965994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21d703-d5c7-4561-865d-1d99ceab6f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.967358 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f43711-9bbb-4400-9d9d-470e4d8d523d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:42.972029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:42.972623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:42.972931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:42.973192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:42.973834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:42.976579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:42.988265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1087/2167] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:42.989550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-3f21d703-d5c7-4561-865d-1d99ceab6f35 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:42.990830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3f21d703-d5c7-4561-865d-1d99ceab6f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:42.998074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.000308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1081/2168] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:43.004625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21d703-d5c7-4561-865d-1d99ceab6f35 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.013299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325ac01-693f-482b-b866-3775df678eb6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.015305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21d703-d5c7-4561-865d-1d99ceab6f35 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.018487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.019192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.019565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.019938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.020284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.020925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.021054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.021205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.021475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.022132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:43.022430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.023143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.024052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.038040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1088/2169] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:43.038879 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.041467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1082/2170] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:00:43.042193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.051451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd79e92-5a55-4acf-94ca-6474c3da3571 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.053316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d274cb-9a0c-4c7c-a624-a97d82a30ae4 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.056272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.057065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.057431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.057794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] PATH_INFO: `/v3/projects/c6d55186b650439c80e9c11d1617776d/users/d84fb2d9d8254775b8c593782d095005/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.058185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471847c2-460a-4999-a1db-e65297a2394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.058786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471847c2-460a-4999-a1db-e65297a2394e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.059102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471847c2-460a-4999-a1db-e65297a2394e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.059319 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471847c2-460a-4999-a1db-e65297a2394e admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.060968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471847c2-460a-4999-a1db-e65297a2394e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.062948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471847c2-460a-4999-a1db-e65297a2394e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.074535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d55186b650439c80e9c11d1617776d, user_id=d84fb2d9d8254775b8c593782d095005, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.077236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.092339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1089/2171] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:43 2022] PUT /identity/v3/projects/c6d55186b650439c80e9c11d1617776d/users/d84fb2d9d8254775b8c593782d095005/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:43.098705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1083/2172] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:43 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:43.103696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.104259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.104259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.104438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.133280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] MFA Rules not processed for user `d84fb2d9d8254775b8c593782d095005`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:43.134481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:43.178005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cd620-edbe-4826-8b92-8652a0fe289e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.221204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1084/2173] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:43.233746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.235402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:43.246560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:43.275020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.275633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.275911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.276187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.276853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.279453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.305428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4d81b-c94c-44ef-bee2-59d2b2c28012 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.307246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1090/2174] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:43 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:43.309302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.310558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.310862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.311181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.311595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.320755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.322210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.322492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5c070-6d6b-4511-8d78-dccde7294ded admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.328536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.329188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.329466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.329737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.330383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.331994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.332561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1085/2175] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:43.343420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1091/2176] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:43.347371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.359301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b23493-70e0-4658-b04a-5fe71629a613 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.363488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.365493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.366105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.366396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.366641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.367333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.368954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.375979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302e162-86b3-46d7-82e6-9ce0fd47ffe5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.381875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.382521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.382820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.383084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] PATH_INFO: `/v3/users/d84fb2d9d8254775b8c593782d095005` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.384368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84fb2d9d8254775b8c593782d095005)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.384899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1086/2177] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:43.385960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.386724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.399476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65e57b-ef8c-4eb4-a5a7-717ba0c7ca1c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.407859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bb470-ebe7-458d-ba13-02cc6157bc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.408534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bb470-ebe7-458d-ba13-02cc6157bc95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.408838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bb470-ebe7-458d-ba13-02cc6157bc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.409130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bb470-ebe7-458d-ba13-02cc6157bc95 admin admin] PATH_INFO: `/v3/projects/789c5aee5f1848989d6ceab773e60007/users/7291dce9e38e42bd91a57a4e68653e61/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.426570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bb470-ebe7-458d-ba13-02cc6157bc95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=789c5aee5f1848989d6ceab773e60007, user_id=7291dce9e38e42bd91a57a4e68653e61, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.429458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bb470-ebe7-458d-ba13-02cc6157bc95 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.441658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:43.447106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1087/2178] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:43 2022] PUT /identity/v3/projects/789c5aee5f1848989d6ceab773e60007/users/7291dce9e38e42bd91a57a4e68653e61/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:43.448122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84fb2d9d8254775b8c593782d095005'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:43.458591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84fb2d9d8254775b8c593782d095005'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:43.459316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.459638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.460370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.474732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84fb2d9d8254775b8c593782d095005'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:43.483179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84fb2d9d8254775b8c593782d095005'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:43.483999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6dcb222b-73f0-45f8-bd52-b7ca5fff5d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84fb2d9d8254775b8c593782d095005'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:43.491866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1092/2179] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:43 2022] DELETE /identity/v3/users/d84fb2d9d8254775b8c593782d095005 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:43.495943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.496389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.496674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.508019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] MFA Rules not processed for user `7291dce9e38e42bd91a57a4e68653e61`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:43.509300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:43.518220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:43.519466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:43.585548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1088/2180] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:43.589547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1093/2181] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:43.597853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.599274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:43.600656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:43.600792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:43.632964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:43.658955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83952b91-be77-4344-b1fe-9e0fc9ce936c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.661832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504ed67-22fd-48f6-bc9f-6a44d1346292 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:43.662748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.663469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.663759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.664079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.665815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adde8e-4af2-43bc-91c0-ae7dbabddbd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:43.666414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adde8e-4af2-43bc-91c0-ae7dbabddbd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:43.666687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adde8e-4af2-43bc-91c0-ae7dbabddbd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:43.666955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adde8e-4af2-43bc-91c0-ae7dbabddbd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:43.673483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.674940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.675539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adde8e-4af2-43bc-91c0-ae7dbabddbd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:43.677205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adde8e-4af2-43bc-91c0-ae7dbabddbd0 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:43.685990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1089/2182] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:43.694103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1094/2183] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:44.039646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.041749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:44.062991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58ed57-166d-4e5f-91a6-a292f2b0020c service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.077358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.078086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.078440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.078788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] PATH_INFO: `/v3/projects/c6d55186b650439c80e9c11d1617776d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.080013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d55186b650439c80e9c11d1617776d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.083021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.144065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d55186b650439c80e9c11d1617776d', 'request_id': 'req-de63b651-ddba-4535-bcbf-88ee95385a3b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.152367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d55186b650439c80e9c11d1617776d', 'request_id': 'req-de63b651-ddba-4535-bcbf-88ee95385a3b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.159856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d55186b650439c80e9c11d1617776d', 'request_id': 'req-de63b651-ddba-4535-bcbf-88ee95385a3b'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.173222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] The token cache is being invalidate because project c6d55186b650439c80e9c11d1617776d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:44.173526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.174175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-de63b651-ddba-4535-bcbf-88ee95385a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.175868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1090/2184] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:44 2022] DELETE /identity/v3/projects/c6d55186b650439c80e9c11d1617776d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:44.180335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee07ad-1c71-4bc3-a361-ca1a3c5d3e1e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.180866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee07ad-1c71-4bc3-a361-ca1a3c5d3e1e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.180952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee07ad-1c71-4bc3-a361-ca1a3c5d3e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.210369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-aeee07ad-1c71-4bc3-a361-ca1a3c5d3e1e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:44.211982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-aeee07ad-1c71-4bc3-a361-ca1a3c5d3e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:44.318758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1095/2185] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:44.329007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed191f52-0363-41f9-9d5f-1d1665d868b1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.329007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed191f52-0363-41f9-9d5f-1d1665d868b1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.329007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed191f52-0363-41f9-9d5f-1d1665d868b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.352243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ed191f52-0363-41f9-9d5f-1d1665d868b1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:44.353750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ed191f52-0363-41f9-9d5f-1d1665d868b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:44.364762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1091/2186] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:44.370793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee07ad-1c71-4bc3-a361-ca1a3c5d3e1e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.381993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee07ad-1c71-4bc3-a361-ca1a3c5d3e1e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.388502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.389173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.389522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.390241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.390493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:44.390806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.392531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.405938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1096/2187] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:00:44.411065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed191f52-0363-41f9-9d5f-1d1665d868b1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.423073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed191f52-0363-41f9-9d5f-1d1665d868b1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.428053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.428639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.428952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.429226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.429873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.431969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.438972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:44.466134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1092/2188] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:44 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:44.471102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.493392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379973f3-6a96-4fdb-848f-04e659c212a7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.554925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.555536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.555811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.556078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.556712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.558639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.584669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1097/2189] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:44 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:44.588502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.601734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0fc30-7196-49cd-a705-d55a53df726d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.609546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.610581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.610880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.611147 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.611802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.614255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.630281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1093/2190] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:44.633453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.645085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33d015-1033-46f9-8456-333b452eb73a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.650026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f02e39-9fb7-4ae4-bae8-648b064e071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.650626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02e39-9fb7-4ae4-bae8-648b064e071c admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.650906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02e39-9fb7-4ae4-bae8-648b064e071c admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.651179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02e39-9fb7-4ae4-bae8-648b064e071c admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.651823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f02e39-9fb7-4ae4-bae8-648b064e071c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.653540 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f02e39-9fb7-4ae4-bae8-648b064e071c admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.668641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1098/2191] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:44.670727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.684155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949e001-1764-46c5-8272-c3600cd6359e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.690255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.691008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.691350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.691688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] PATH_INFO: `/v3/projects/afe649b53db84533af6093e7f7207c18/users/0f5317cfdeb6462bbb2091fb99e54597/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.708102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe649b53db84533af6093e7f7207c18, user_id=0f5317cfdeb6462bbb2091fb99e54597, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.710857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.732752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1094/2192] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:44 2022] PUT /identity/v3/projects/afe649b53db84533af6093e7f7207c18/users/0f5317cfdeb6462bbb2091fb99e54597/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:44.744017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.744286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.744590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.775803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.777582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:44.781317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] MFA Rules not processed for user `0f5317cfdeb6462bbb2091fb99e54597`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:44.782662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:44.814818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:44.850736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8f3de-ad10-4f45-91a3-6b6fa36f4737 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.855475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.856111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.856526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.856925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] PATH_INFO: `/v3/users/088796cfb8bc4d0693a1c16f63399320` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.858088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=088796cfb8bc4d0693a1c16f63399320)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.858923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1099/2193] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:44.860119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.869469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:44.870957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:44.918602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088796cfb8bc4d0693a1c16f63399320'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.919570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088796cfb8bc4d0693a1c16f63399320'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.928222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088796cfb8bc4d0693a1c16f63399320'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.942151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdece38a-fc02-4286-8466-72e8de48356a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:44.945384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088796cfb8bc4d0693a1c16f63399320'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.946120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:44.946742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.947015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.947284 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.949115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:44.954364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2a9a179e-a807-4404-917c-a91608ede5fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088796cfb8bc4d0693a1c16f63399320'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:44.961176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1095/2194] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:44 2022] DELETE /identity/v3/users/088796cfb8bc4d0693a1c16f63399320 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:44.962201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:44.963636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:44.965868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:44.966149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:44.966549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:44.973968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1100/2195] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:44.988353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:44.989605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:45.024599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:45.080835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1096/2196] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:45.090825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:45.092923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:45.162374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c0c94-3103-4e30-b4f2-073d29a3ac64 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:45.227376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:45.228393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:45.228511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:45.228927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:45.240943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:45.242934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:45.256416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1101/2197] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:45.584288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:45.585972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:45.599955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a76c8-8184-4454-9c6e-a7142c79ca0c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:45.604953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:45.605613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:45.606087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:45.607117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] PATH_INFO: `/v3/projects/58bab1a9aecb48a4a7b18d6f7f88f74c` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:45.608291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58bab1a9aecb48a4a7b18d6f7f88f74c)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:45.610254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:45.657674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58bab1a9aecb48a4a7b18d6f7f88f74c', 'request_id': 'req-3b56b0dd-7752-4aea-8b4c-766b324b64c5'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.665033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58bab1a9aecb48a4a7b18d6f7f88f74c', 'request_id': 'req-3b56b0dd-7752-4aea-8b4c-766b324b64c5'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.671244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58bab1a9aecb48a4a7b18d6f7f88f74c', 'request_id': 'req-3b56b0dd-7752-4aea-8b4c-766b324b64c5'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.682033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] The token cache is being invalidate because project 58bab1a9aecb48a4a7b18d6f7f88f74c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:45.682321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.682940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.684454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1097/2198] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:45 2022] DELETE /identity/v3/projects/58bab1a9aecb48a4a7b18d6f7f88f74c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:45.688402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:45.697978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dea4d1-3b33-4646-8354-6152cc139459 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:45.725009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:45.755999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:45.756621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:45.756945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:45.757198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] PATH_INFO: `/v3/users/b960f3ec76044d6a96b70baa1da80463` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:45.758317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b960f3ec76044d6a96b70baa1da80463)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:45.760297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:45.810005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b960f3ec76044d6a96b70baa1da80463'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.818215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b960f3ec76044d6a96b70baa1da80463'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.833101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b960f3ec76044d6a96b70baa1da80463'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.843923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b960f3ec76044d6a96b70baa1da80463'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.844669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b960f3ec76044d6a96b70baa1da80463'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:45.851165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1102/2199] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:45 2022] DELETE /identity/v3/users/b960f3ec76044d6a96b70baa1da80463 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:46.130883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:46.132283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:46.147755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56b0dd-7752-4aea-8b4c-766b324b64c5 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:46.212515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:46.213197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:46.213455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:46.213726 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] PATH_INFO: `/v3/projects/508b352baaff4fe0a055023d6fc8a2d6` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:46.214858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=508b352baaff4fe0a055023d6fc8a2d6)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:46.217978 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:46.266955 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '508b352baaff4fe0a055023d6fc8a2d6', 'request_id': 'req-c5d5ffdc-17a6-4a93-9609-f222324e90c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.275167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '508b352baaff4fe0a055023d6fc8a2d6', 'request_id': 'req-c5d5ffdc-17a6-4a93-9609-f222324e90c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.277181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:46.285107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '508b352baaff4fe0a055023d6fc8a2d6', 'request_id': 'req-c5d5ffdc-17a6-4a93-9609-f222324e90c7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.297691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] The token cache is being invalidate because project 508b352baaff4fe0a055023d6fc8a2d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:46.297977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.298650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.300756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1098/2200] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:46 2022] DELETE /identity/v3/projects/508b352baaff4fe0a055023d6fc8a2d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:46.305301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:46.306738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:46.320976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf37ea3-35f2-4714-91c4-7e244218a2f1 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:46.407094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:46.407764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:46.408043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:46.408314 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] PATH_INFO: `/v3/users/a961968a9bd1493b823dd5296596f640` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:46.409995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a961968a9bd1493b823dd5296596f640)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:46.412123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:46.469780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a961968a9bd1493b823dd5296596f640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.472325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:46.478175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a961968a9bd1493b823dd5296596f640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.501837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a961968a9bd1493b823dd5296596f640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.510736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a961968a9bd1493b823dd5296596f640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.511479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f544b0d9-52fd-4e6f-b10f-c7a4bdfcf658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a961968a9bd1493b823dd5296596f640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:46.518776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1103/2201] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:46 2022] DELETE /identity/v3/users/a961968a9bd1493b823dd5296596f640 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:46.827339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:46.828830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:46.845409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5ffdc-17a6-4a93-9609-f222324e90c7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:46.919640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:46.920477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:46.920802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:46.921081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] PATH_INFO: `/v3/projects/95912f4cf6964f13a6ba636bf133ffda` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:46.922244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95912f4cf6964f13a6ba636bf133ffda)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:46.924192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:46.993665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95912f4cf6964f13a6ba636bf133ffda', 'request_id': 'req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:47.005278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95912f4cf6964f13a6ba636bf133ffda', 'request_id': 'req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:47.012763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95912f4cf6964f13a6ba636bf133ffda', 'request_id': 'req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:47.027509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] The token cache is being invalidate because project 95912f4cf6964f13a6ba636bf133ffda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:47.027799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:47.029729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6ce70522-4b22-4ed0-88aa-dbdd61bb68a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:47.031289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1099/2202] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:46 2022] DELETE /identity/v3/projects/95912f4cf6964f13a6ba636bf133ffda => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:47.040530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536b52-aafd-4b67-898a-2ef7933af8c0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.040746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536b52-aafd-4b67-898a-2ef7933af8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.041164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb536b52-aafd-4b67-898a-2ef7933af8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.065527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-cb536b52-aafd-4b67-898a-2ef7933af8c0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:47.066751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cb536b52-aafd-4b67-898a-2ef7933af8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:47.138174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1104/2203] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:47.148131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628928d5-3441-461d-95d7-9323443bd036 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.148445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628928d5-3441-461d-95d7-9323443bd036 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.148749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628928d5-3441-461d-95d7-9323443bd036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.174307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-628928d5-3441-461d-95d7-9323443bd036 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:47.175541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-628928d5-3441-461d-95d7-9323443bd036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:47.183749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1100/2204] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:47.189287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536b52-aafd-4b67-898a-2ef7933af8c0 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.201214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb536b52-aafd-4b67-898a-2ef7933af8c0 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.210952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.211560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.211847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.212096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.212763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:47.213066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.214755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.233567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1105/2205] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:00:47.238063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628928d5-3441-461d-95d7-9323443bd036 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.250436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628928d5-3441-461d-95d7-9323443bd036 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.255738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.256826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.257202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.257478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.258131 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.260268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.295741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1101/2206] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:47 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:47.304792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.323922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f7dde-d25b-4917-8120-dc144b92d5a6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.385654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.386264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.386542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.386812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.388106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.390369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.417738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1106/2207] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:47 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:47.420583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.434115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ea2d8-1e3e-471f-9afc-6caffb389589 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.440457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.441428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.441728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.442264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.443199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.456843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.457348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1102/2208] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:47.460907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.472097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08d3e9-da1b-4e19-aada-a860e5c7f8aa admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.477107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85429b-bf19-4ac9-ab21-96cf2593bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.477709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85429b-bf19-4ac9-ab21-96cf2593bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.477989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85429b-bf19-4ac9-ab21-96cf2593bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.478286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85429b-bf19-4ac9-ab21-96cf2593bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.478930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85429b-bf19-4ac9-ab21-96cf2593bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.480537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85429b-bf19-4ac9-ab21-96cf2593bab4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.491993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1107/2209] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:47.496197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.506769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb078006-4775-43c1-aa01-ce74d5097da6 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.511397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.512034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.512314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.512740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] PATH_INFO: `/v3/projects/cb8566715e034f2ebb6dd84c96fbec60/users/888a96f1dd6b44cfb76606708f4f95fc/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.527014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8566715e034f2ebb6dd84c96fbec60, user_id=888a96f1dd6b44cfb76606708f4f95fc, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.529847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.544769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1103/2210] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:47 2022] PUT /identity/v3/projects/cb8566715e034f2ebb6dd84c96fbec60/users/888a96f1dd6b44cfb76606708f4f95fc/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:47.555710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f60264-2e3c-46e0-8051-34e53ddf78ee None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.555992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f60264-2e3c-46e0-8051-34e53ddf78ee None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.556264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f60264-2e3c-46e0-8051-34e53ddf78ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.584814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b5f60264-2e3c-46e0-8051-34e53ddf78ee None None] MFA Rules not processed for user `888a96f1dd6b44cfb76606708f4f95fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:47.586207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b5f60264-2e3c-46e0-8051-34e53ddf78ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:47.608076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:47.654887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1108/2211] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:47.659448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.672528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:47.723353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496edc40-1579-48bc-92c7-d03f5c2674e7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.728169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.728795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.729068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.729334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.729968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.732538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.758989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1104/2212] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:47 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:47.763337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f60264-2e3c-46e0-8051-34e53ddf78ee None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.773999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f60264-2e3c-46e0-8051-34e53ddf78ee None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.832043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.832643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.832949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.833221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.833853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.835777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.858466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1109/2213] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:47 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:47.862500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.873198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1961811-3914-4006-880e-4683559d2264 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.877827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.878421 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.878697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.878975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.879616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.881267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.892573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1105/2214] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:47.896692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.907414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853839f-3c29-40d1-a4e3-3a3474b160a5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:47.912472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:47.913172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:47.913447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:47.913715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] PATH_INFO: `/v3/projects/0adbea5650774bb7b42fa074cad120e0/users/7159080725e54d58840f3de80ca7d9b3/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:47.930079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0adbea5650774bb7b42fa074cad120e0, user_id=7159080725e54d58840f3de80ca7d9b3, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:47.932872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:47.945563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1110/2215] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:47 2022] PUT /identity/v3/projects/0adbea5650774bb7b42fa074cad120e0/users/7159080725e54d58840f3de80ca7d9b3/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:47.949637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:47.960299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f66cf-012f-47fb-ae09-dc9eacc0ba21 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:48.017819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481cfe4-be39-4341-89de-e34f672cd49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:48.018426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481cfe4-be39-4341-89de-e34f672cd49f admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:48.018702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481cfe4-be39-4341-89de-e34f672cd49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:48.018969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481cfe4-be39-4341-89de-e34f672cd49f admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:48.019610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481cfe4-be39-4341-89de-e34f672cd49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:48.021252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481cfe4-be39-4341-89de-e34f672cd49f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:48.031597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1106/2216] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:48.035756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:48.046480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdba3d9-eba2-4742-be88-127b68dccd92 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:48.051123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:48.051767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:48.052045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:48.052316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] PATH_INFO: `/v3/projects/0adbea5650774bb7b42fa074cad120e0/users/7159080725e54d58840f3de80ca7d9b3/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:48.054242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0adbea5650774bb7b42fa074cad120e0, user_id=7159080725e54d58840f3de80ca7d9b3, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:48.056910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:48.067099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1111/2217] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:48 2022] PUT /identity/v3/projects/0adbea5650774bb7b42fa074cad120e0/users/7159080725e54d58840f3de80ca7d9b3/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:48.077591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490fc2f-f403-4741-8101-c542e7235692 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:48.077904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490fc2f-f403-4741-8101-c542e7235692 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:48.078178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490fc2f-f403-4741-8101-c542e7235692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:48.107434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-4490fc2f-f403-4741-8101-c542e7235692 None None] MFA Rules not processed for user `7159080725e54d58840f3de80ca7d9b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:48.108553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-4490fc2f-f403-4741-8101-c542e7235692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:48.176884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1107/2218] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:48.188278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:48.189997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:48.250058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f4ffe-0ec4-45e7-9279-32c94e3da4b2 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:48.253735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8071d40-ce5d-40b7-ac6d-b9b43b2d8f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:48.254340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071d40-ce5d-40b7-ac6d-b9b43b2d8f34 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:48.254617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071d40-ce5d-40b7-ac6d-b9b43b2d8f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:48.254885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8071d40-ce5d-40b7-ac6d-b9b43b2d8f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:48.265208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8071d40-ce5d-40b7-ac6d-b9b43b2d8f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:48.266644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8071d40-ce5d-40b7-ac6d-b9b43b2d8f34 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:48.276737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1112/2219] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:48.424115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490fc2f-f403-4741-8101-c542e7235692 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:48.432841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:48.500805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490fc2f-f403-4741-8101-c542e7235692 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:48.506729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:48.507444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:48.507860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:48.508248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] PATH_INFO: `/v3/users/888a96f1dd6b44cfb76606708f4f95fc` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:48.509509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=888a96f1dd6b44cfb76606708f4f95fc)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:48.511514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:48.574416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888a96f1dd6b44cfb76606708f4f95fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:48.575972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888a96f1dd6b44cfb76606708f4f95fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:48.586447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888a96f1dd6b44cfb76606708f4f95fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:48.601639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888a96f1dd6b44cfb76606708f4f95fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:48.615834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888a96f1dd6b44cfb76606708f4f95fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:48.624943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1108/2220] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:48 2022] DELETE /identity/v3/users/888a96f1dd6b44cfb76606708f4f95fc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:48.630212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:48.630681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:48.630866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:48.652080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:48.653739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:48.666266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:48.716068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1113/2221] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:48.725920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:48.727315 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:48.785641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d8dcd-b4fb-4cfb-be8f-9daf035b8240 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:48.789233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:48.789830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:48.790106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:48.790375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:48.799249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:48.800735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:48.812050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1109/2222] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:49.106555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:49.108080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:49.123290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83b8d-a223-4710-acb3-b31810060795 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:49.128565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:49.129143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:49.129367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:49.129636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] PATH_INFO: `/v3/projects/cb8566715e034f2ebb6dd84c96fbec60` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:49.130811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8566715e034f2ebb6dd84c96fbec60)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:49.132714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:49.184204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8566715e034f2ebb6dd84c96fbec60', 'request_id': 'req-f08c7216-71f0-44c1-8b37-14eda33dcd73'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.192815 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8566715e034f2ebb6dd84c96fbec60', 'request_id': 'req-f08c7216-71f0-44c1-8b37-14eda33dcd73'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.198445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8566715e034f2ebb6dd84c96fbec60', 'request_id': 'req-f08c7216-71f0-44c1-8b37-14eda33dcd73'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.210007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] The token cache is being invalidate because project cb8566715e034f2ebb6dd84c96fbec60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:49.210295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.211345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.213002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1114/2223] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:49 2022] DELETE /identity/v3/projects/cb8566715e034f2ebb6dd84c96fbec60 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:49.217920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:49.227822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5603d-d28f-4180-8931-c82432221b24 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:49.253380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:49.287832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:49.288455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:49.288747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:49.289039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] PATH_INFO: `/v3/users/7159080725e54d58840f3de80ca7d9b3` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:49.290164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7159080725e54d58840f3de80ca7d9b3)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:49.292099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:49.340143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7159080725e54d58840f3de80ca7d9b3'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.341183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7159080725e54d58840f3de80ca7d9b3'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.348452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7159080725e54d58840f3de80ca7d9b3'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.360307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7159080725e54d58840f3de80ca7d9b3'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.367067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-cfc932c3-9dd3-48f5-97e2-7c0dce932273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7159080725e54d58840f3de80ca7d9b3'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:49.373870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1110/2224] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:49 2022] DELETE /identity/v3/users/7159080725e54d58840f3de80ca7d9b3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:49.793470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:49.795105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:49.814961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c7216-71f0-44c1-8b37-14eda33dcd73 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:49.925656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:49.926549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:49.926825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:49.927099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] PATH_INFO: `/v3/projects/0adbea5650774bb7b42fa074cad120e0` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:49.928471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0adbea5650774bb7b42fa074cad120e0)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:49.930549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:49.987065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:49.991624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0adbea5650774bb7b42fa074cad120e0', 'request_id': 'req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:50.000359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0adbea5650774bb7b42fa074cad120e0', 'request_id': 'req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:50.009149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0adbea5650774bb7b42fa074cad120e0', 'request_id': 'req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:50.021866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] The token cache is being invalidate because project 0adbea5650774bb7b42fa074cad120e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:50.022190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:50.022908 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a67ae4c5-1063-4bb6-9c9c-3dc638ba2a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:50.024550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1115/2225] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:49 2022] DELETE /identity/v3/projects/0adbea5650774bb7b42fa074cad120e0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:50.030079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b551e35-5020-4d4b-933e-1e24e64f7b2b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.030370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b551e35-5020-4d4b-933e-1e24e64f7b2b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.030645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b551e35-5020-4d4b-933e-1e24e64f7b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.055008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-3b551e35-5020-4d4b-933e-1e24e64f7b2b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:50.056401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3b551e35-5020-4d4b-933e-1e24e64f7b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:50.150416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1111/2226] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:50.158318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e38b0b-afc2-44a1-a7cd-dc43a4f84615 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.158983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e38b0b-afc2-44a1-a7cd-dc43a4f84615 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.158983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e38b0b-afc2-44a1-a7cd-dc43a4f84615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.199416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a2e38b0b-afc2-44a1-a7cd-dc43a4f84615 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:50.200095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a2e38b0b-afc2-44a1-a7cd-dc43a4f84615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:50.210764 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1116/2227] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:50.216475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b551e35-5020-4d4b-933e-1e24e64f7b2b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.229455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b551e35-5020-4d4b-933e-1e24e64f7b2b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.235475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.236766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.236766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.237111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.238040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:50.238447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.240259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.257026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1112/2228] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:50.272758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e38b0b-afc2-44a1-a7cd-dc43a4f84615 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.284175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e38b0b-afc2-44a1-a7cd-dc43a4f84615 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.290561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.291402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.291884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.292299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.293831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.296028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.333530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1117/2229] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:50 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:50.338416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.353965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af5a33-d497-454c-93d1-f19bd2558899 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.387986 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:50.415471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.416085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.416366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.416640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.417320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.420630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.447352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1113/2230] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:50 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:50.451848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.462297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2d363-8653-4476-acea-9a0e17dbc52e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.468014 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.468612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.468935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.469204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.469848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.471449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.482043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1118/2231] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:50.486048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.495868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80d909-0e2e-4bf7-842f-51e4c6c62593 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.500896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde0e4b-f1e7-498a-aa9e-a0d2f7343edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.501492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0e4b-f1e7-498a-aa9e-a0d2f7343edb admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.501764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0e4b-f1e7-498a-aa9e-a0d2f7343edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.502029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde0e4b-f1e7-498a-aa9e-a0d2f7343edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.502853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0e4b-f1e7-498a-aa9e-a0d2f7343edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.504284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde0e4b-f1e7-498a-aa9e-a0d2f7343edb admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.517168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1114/2232] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:50.521156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.532453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472e97-7213-4602-b69d-e0c820bca84e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.538531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.539601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.540032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.541170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] PATH_INFO: `/v3/projects/401e199be6774c538468d21ef54edff2/users/c0f0b9f44e264a3995e57ba25f97a4e8/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.560438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401e199be6774c538468d21ef54edff2, user_id=c0f0b9f44e264a3995e57ba25f97a4e8, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.563161 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.584817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1119/2233] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:50 2022] PUT /identity/v3/projects/401e199be6774c538468d21ef54edff2/users/c0f0b9f44e264a3995e57ba25f97a4e8/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:50.592790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bec690-4e76-4628-b4cc-268bdd2efbf1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.593578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bec690-4e76-4628-b4cc-268bdd2efbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.593865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bec690-4e76-4628-b4cc-268bdd2efbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.632934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e2bec690-4e76-4628-b4cc-268bdd2efbf1 None None] MFA Rules not processed for user `c0f0b9f44e264a3995e57ba25f97a4e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:50.644669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e2bec690-4e76-4628-b4cc-268bdd2efbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:50.733071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1115/2234] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:50.734393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.817084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c6d9d-40b6-4e72-982a-e7f490c363d6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.822108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.822808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.823147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.823481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.824186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.827078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.889288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1120/2235] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:50 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:50.890449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bec690-4e76-4628-b4cc-268bdd2efbf1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:50.902148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bec690-4e76-4628-b4cc-268bdd2efbf1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:50.967450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:50.968061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:50.968339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:50.968611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:50.969284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:50.971219 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:50.995100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1116/2236] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:50 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:50.999291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:51.010811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233700b7-ee16-4072-bfb5-250c15cf0df7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:51.016055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:51.016787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:51.017133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:51.017476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:51.018198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:51.019798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:51.031603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1121/2237] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:51.035599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:51.046544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115ad72-7491-4713-846e-750495cc9a4d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:51.051197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:51.051847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:51.052128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:51.052401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] PATH_INFO: `/v3/projects/94afc4ddfa0b455a8250b45198ae4013/users/65f6039a56d24139a924352282943df8/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:51.068933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94afc4ddfa0b455a8250b45198ae4013, user_id=65f6039a56d24139a924352282943df8, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:51.071687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:51.084946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1117/2238] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:51 2022] PUT /identity/v3/projects/94afc4ddfa0b455a8250b45198ae4013/users/65f6039a56d24139a924352282943df8/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:51.089190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:51.101011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa578c-159a-4b6e-81d8-0a32f72df66f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:51.148058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:51.179818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d37c8-2c6d-4957-94f8-9aeddf3fed8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:51.180424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d37c8-2c6d-4957-94f8-9aeddf3fed8e admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:51.180700 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d37c8-2c6d-4957-94f8-9aeddf3fed8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:51.181013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d37c8-2c6d-4957-94f8-9aeddf3fed8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:51.181656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d37c8-2c6d-4957-94f8-9aeddf3fed8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:51.183261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d37c8-2c6d-4957-94f8-9aeddf3fed8e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:51.196735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1122/2239] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:51.201674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:51.213283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa0705-8a1b-4609-8aad-d5f736e8c880 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:51.219833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:51.220621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:51.220992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:51.221302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] PATH_INFO: `/v3/projects/94afc4ddfa0b455a8250b45198ae4013/users/65f6039a56d24139a924352282943df8/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:51.223347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94afc4ddfa0b455a8250b45198ae4013, user_id=65f6039a56d24139a924352282943df8, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:51.226167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:51.229767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:51.240550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1118/2240] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:51 2022] PUT /identity/v3/projects/94afc4ddfa0b455a8250b45198ae4013/users/65f6039a56d24139a924352282943df8/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:51.252215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10d5f-f733-4c32-ba54-7deffc0c57c2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:51.252588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10d5f-f733-4c32-ba54-7deffc0c57c2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:51.252962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d10d5f-f733-4c32-ba54-7deffc0c57c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:51.282005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-85d10d5f-f733-4c32-ba54-7deffc0c57c2 None None] MFA Rules not processed for user `65f6039a56d24139a924352282943df8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:51.283213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-85d10d5f-f733-4c32-ba54-7deffc0c57c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:51.366667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1123/2241] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:51.379534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:51.381320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:51.449660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c31cd-fd9f-4e0f-9755-6d6a7e257287 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:51.455216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f00aa-ffc3-4fc1-aad2-0507472ed08b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:51.455841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f00aa-ffc3-4fc1-aad2-0507472ed08b service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:51.456126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f00aa-ffc3-4fc1-aad2-0507472ed08b service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:51.456395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f00aa-ffc3-4fc1-aad2-0507472ed08b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:51.466372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f00aa-ffc3-4fc1-aad2-0507472ed08b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:51.468263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f00aa-ffc3-4fc1-aad2-0507472ed08b service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:51.479933 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1119/2242] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:51.845423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10d5f-f733-4c32-ba54-7deffc0c57c2 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:51.907893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d10d5f-f733-4c32-ba54-7deffc0c57c2 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:51.912416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:51.913090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:51.913368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:51.913644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] PATH_INFO: `/v3/users/c0f0b9f44e264a3995e57ba25f97a4e8` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:51.914775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f0b9f44e264a3995e57ba25f97a4e8)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:51.916705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:51.987008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f0b9f44e264a3995e57ba25f97a4e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:51.995452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f0b9f44e264a3995e57ba25f97a4e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.007236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:52.013655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f0b9f44e264a3995e57ba25f97a4e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.023091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f0b9f44e264a3995e57ba25f97a4e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.023841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f0b9f44e264a3995e57ba25f97a4e8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.031435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1124/2243] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:51 2022] DELETE /identity/v3/users/c0f0b9f44e264a3995e57ba25f97a4e8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:52.036070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:52.036382 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:52.036760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:52.057312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:52.058547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:52.125364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1120/2244] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:52.135111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:52.136520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:52.204520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba196f-8cc2-49c5-8f7c-8702967d53e6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:52.208278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:52.208917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:52.209200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:52.209471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:52.218717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:52.220158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:52.229813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1125/2245] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:52.517791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:52.519235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:52.534058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00d6d4-b428-44e4-8a7c-561df84cae19 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:52.538638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:52.539757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:52.540208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:52.540481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] PATH_INFO: `/v3/projects/401e199be6774c538468d21ef54edff2` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:52.541893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=401e199be6774c538468d21ef54edff2)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:52.544301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:52.589805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401e199be6774c538468d21ef54edff2', 'request_id': 'req-d3fccdc7-33e1-420d-ba80-415a42c50063'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.597932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401e199be6774c538468d21ef54edff2', 'request_id': 'req-d3fccdc7-33e1-420d-ba80-415a42c50063'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.603552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401e199be6774c538468d21ef54edff2', 'request_id': 'req-d3fccdc7-33e1-420d-ba80-415a42c50063'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.616067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] The token cache is being invalidate because project 401e199be6774c538468d21ef54edff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:52.616352 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.617014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.618769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1121/2246] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:52 2022] DELETE /identity/v3/projects/401e199be6774c538468d21ef54edff2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:52.623153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:52.633132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846bac5-9899-4c39-8ebb-cffccb8c9208 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:52.689662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:52.690282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:52.690575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:52.690830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] PATH_INFO: `/v3/users/65f6039a56d24139a924352282943df8` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:52.691943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f6039a56d24139a924352282943df8)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:52.693889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:52.739229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f6039a56d24139a924352282943df8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.746333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f6039a56d24139a924352282943df8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.758349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f6039a56d24139a924352282943df8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.765944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f6039a56d24139a924352282943df8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.766571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-47846727-f4d7-4aa3-b39f-46f529d4841d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f6039a56d24139a924352282943df8'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:52.772335 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1126/2247] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:52 2022] DELETE /identity/v3/users/65f6039a56d24139a924352282943df8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:53.050624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:53.053797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:53.072513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fccdc7-33e1-420d-ba80-415a42c50063 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:53.095538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:53.139110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:53.139788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.140065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.140350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] PATH_INFO: `/v3/projects/94afc4ddfa0b455a8250b45198ae4013` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.141723 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94afc4ddfa0b455a8250b45198ae4013)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:53.143701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:53.193458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94afc4ddfa0b455a8250b45198ae4013', 'request_id': 'req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:53.202375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94afc4ddfa0b455a8250b45198ae4013', 'request_id': 'req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:53.211322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94afc4ddfa0b455a8250b45198ae4013', 'request_id': 'req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:53.226657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] The token cache is being invalidate because project 94afc4ddfa0b455a8250b45198ae4013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:53.227047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:53.227919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2f63f05c-10b6-4a4a-95dd-d2c1ef3b6d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:53.232813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1122/2248] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:53 2022] DELETE /identity/v3/projects/94afc4ddfa0b455a8250b45198ae4013 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:53.234578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ce549-2e32-4dc8-8e84-265ee1350ec2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.235094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ce549-2e32-4dc8-8e84-265ee1350ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.235184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ce549-2e32-4dc8-8e84-265ee1350ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.244944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:53.258899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-9e8ce549-2e32-4dc8-8e84-265ee1350ec2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:53.263007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9e8ce549-2e32-4dc8-8e84-265ee1350ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:53.393509 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1127/2249] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:53.401378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b64dd-8869-48bc-b1f0-8a60e70c45a9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.401719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b64dd-8869-48bc-b1f0-8a60e70c45a9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.402092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b64dd-8869-48bc-b1f0-8a60e70c45a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.430184 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-993b64dd-8869-48bc-b1f0-8a60e70c45a9 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:53.431542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-993b64dd-8869-48bc-b1f0-8a60e70c45a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:53.441773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1123/2250] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:53.446246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ce549-2e32-4dc8-8e84-265ee1350ec2 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:53.476901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ce549-2e32-4dc8-8e84-265ee1350ec2 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:53.483309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:53.483964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.484241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.484504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.485867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:53.485867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:53.487146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:53.503650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1128/2251] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:00:53.509236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b64dd-8869-48bc-b1f0-8a60e70c45a9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:53.521098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b64dd-8869-48bc-b1f0-8a60e70c45a9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:53.533880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:53.534569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.534855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.535130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.535841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:53.538762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:53.572830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1124/2252] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:53 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:53.576572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:53.587281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1118a0-b3a3-4760-80f6-28f5e0266fbb admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:53.651742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:53.652223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.652655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.653016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.653681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:53.656103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:53.680562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1129/2253] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:53 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:53.685462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:53.697224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56896f5a-0edb-40e0-8851-7a7e2322e35e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:53.702441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:53.703044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.704408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.704897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.705916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:53.708009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:53.724380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1125/2254] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:53.726623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:53.744085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324bd49-b859-45a2-b5f6-21c67b7bceb1 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:53.751739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fef427-b189-437c-8a4f-f1659f7444ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:53.752987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fef427-b189-437c-8a4f-f1659f7444ad admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.753271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fef427-b189-437c-8a4f-f1659f7444ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.758929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fef427-b189-437c-8a4f-f1659f7444ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.759598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fef427-b189-437c-8a4f-f1659f7444ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:53.761485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fef427-b189-437c-8a4f-f1659f7444ad admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:53.773936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1130/2255] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:53.779133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:53.793005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b6854-de02-4260-b2c2-ad81f8d0109f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:53.798785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:53.799461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.799741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.800017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] PATH_INFO: `/v3/projects/aa0dacd264f74bb5a09a8adce00ba20d/users/bf3d9976db2f4f7c80f3db0c0095f427/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.816457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0dacd264f74bb5a09a8adce00ba20d, user_id=bf3d9976db2f4f7c80f3db0c0095f427, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:53.819288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:53.834061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1126/2256] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:53 2022] PUT /identity/v3/projects/aa0dacd264f74bb5a09a8adce00ba20d/users/bf3d9976db2f4f7c80f3db0c0095f427/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:53.845786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0d256-7482-49d6-a784-c4361251b27a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:53.846075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0d256-7482-49d6-a784-c4361251b27a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:53.846355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0d256-7482-49d6-a784-c4361251b27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:53.879251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-69c0d256-7482-49d6-a784-c4361251b27a None None] MFA Rules not processed for user `bf3d9976db2f4f7c80f3db0c0095f427`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:53.880484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-69c0d256-7482-49d6-a784-c4361251b27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:53.947826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:53.955085 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1131/2257] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:53.956534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.010489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:54.027082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e01f3-9690-4f5b-a9a7-1c38346d005a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.034136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.034819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.035240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.035576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.036324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.038414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.069970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1127/2258] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:53 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:54.071077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0d256-7482-49d6-a784-c4361251b27a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.082759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0d256-7482-49d6-a784-c4361251b27a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.144753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.145370 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.145645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.145918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.146573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.148491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.170702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1132/2259] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:54 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:54.174657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.186446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0d92f-2279-4370-a944-e12a425d338e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.192448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.193306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.193426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.194233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.194946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.196495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.209359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1128/2260] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:54.213519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.224216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ea94e-5faf-400c-a2c6-57f7243ca39a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.229215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.229862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.230143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.230415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] PATH_INFO: `/v3/projects/38d54311285c47f9b5d1ce8d2969f136/users/91987b28394c491095ae134f516badc2/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.246275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d54311285c47f9b5d1ce8d2969f136, user_id=91987b28394c491095ae134f516badc2, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.250152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.263285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1133/2261] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:54 2022] PUT /identity/v3/projects/38d54311285c47f9b5d1ce8d2969f136/users/91987b28394c491095ae134f516badc2/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:54.267277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.277301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e2a64-cb6d-48d9-a2cc-726c19982cbb admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.334702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7aa8-7c22-4336-b4b7-f8f967f4e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.335317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7aa8-7c22-4336-b4b7-f8f967f4e9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.335589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7aa8-7c22-4336-b4b7-f8f967f4e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.335857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7aa8-7c22-4336-b4b7-f8f967f4e9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.336500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc7aa8-7c22-4336-b4b7-f8f967f4e9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.338135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc7aa8-7c22-4336-b4b7-f8f967f4e9d4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.348251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1129/2262] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:54.352442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.364776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9a3d7-d750-451d-bdf6-98a0414edb67 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.369417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.370068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.370347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.370626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] PATH_INFO: `/v3/projects/38d54311285c47f9b5d1ce8d2969f136/users/91987b28394c491095ae134f516badc2/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.373202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d54311285c47f9b5d1ce8d2969f136, user_id=91987b28394c491095ae134f516badc2, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.375850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.386321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1134/2263] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:54 2022] PUT /identity/v3/projects/38d54311285c47f9b5d1ce8d2969f136/users/91987b28394c491095ae134f516badc2/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:54.399671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.399952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.400222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.428527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] MFA Rules not processed for user `91987b28394c491095ae134f516badc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:54.429683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:54.511423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1130/2264] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:54.524346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.526949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:54.588043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.589681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:54.596673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694ac88-3aa9-47d4-bb44-73aeb324386e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.601034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.601752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.602147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.602651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.616954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.618406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.629636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1135/2265] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:54.664923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:54.686319 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15eb0b-edad-4373-8bd4-6506604734ef None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.691041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.691670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.691942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.692208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] PATH_INFO: `/v3/users/7291dce9e38e42bd91a57a4e68653e61` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.693530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7291dce9e38e42bd91a57a4e68653e61)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.695657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.718174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.727440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c650e-15ec-4fbf-bfd5-f6ba076cd630 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:54.731903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:54.732527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.732841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.733114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.745242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7291dce9e38e42bd91a57a4e68653e61'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:54.745893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7291dce9e38e42bd91a57a4e68653e61'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:54.755106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7291dce9e38e42bd91a57a4e68653e61'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:54.767748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7291dce9e38e42bd91a57a4e68653e61'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:54.775185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-641eab5c-6cc0-48fd-9144-7f9adc2b53cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7291dce9e38e42bd91a57a4e68653e61'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:54.781408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1131/2266] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:54 2022] DELETE /identity/v3/users/7291dce9e38e42bd91a57a4e68653e61 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:54.785719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:54.786029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:54.786295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:54.789289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:54.794292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:54.796183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:54.805340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1136/2267] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:00:54.805769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:54.807082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:54.877629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1132/2268] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:54.885170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:54.886619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:54.950701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc43a7a-34cc-419d-a0b8-05ab51da5d05 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.003885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f757e5-e0b9-4c8d-b455-e85ab3f4b121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.004650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f757e5-e0b9-4c8d-b455-e85ab3f4b121 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.005062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f757e5-e0b9-4c8d-b455-e85ab3f4b121 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.005467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f757e5-e0b9-4c8d-b455-e85ab3f4b121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.023898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f757e5-e0b9-4c8d-b455-e85ab3f4b121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.025447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f757e5-e0b9-4c8d-b455-e85ab3f4b121 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.036807 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1137/2269] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:55.090891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.092700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:55.107493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1c28d-e055-4035-9f22-9f99a36ee1da None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.112959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.115164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.115469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.115929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] PATH_INFO: `/v3/users/bf3d9976db2f4f7c80f3db0c0095f427` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.117496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3d9976db2f4f7c80f3db0c0095f427)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.119884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.174765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3d9976db2f4f7c80f3db0c0095f427'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.175420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3d9976db2f4f7c80f3db0c0095f427'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.196962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3d9976db2f4f7c80f3db0c0095f427'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.196962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:55.206642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3d9976db2f4f7c80f3db0c0095f427'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.215744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3d9976db2f4f7c80f3db0c0095f427'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.225143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1133/2270] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:55 2022] DELETE /identity/v3/users/bf3d9976db2f4f7c80f3db0c0095f427 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:55.229542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.229850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.230126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.253469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:55.254757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:55.318336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1138/2271] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:55.328571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.330238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:55.349593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.351028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:55.369455 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e169e33-6ee3-4970-86ea-d0df4efb3e88 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.374752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.375511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.376246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.376246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] PATH_INFO: `/v3/projects/789c5aee5f1848989d6ceab773e60007` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.377526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] RBAC: Authorizing `identity:delete_project(project_id=789c5aee5f1848989d6ceab773e60007)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.379893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.395313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a18f5-4a35-4462-b981-284952a77ff7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.399010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb01ca-f7a6-4544-a12f-2097d8779112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.399594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb01ca-f7a6-4544-a12f-2097d8779112 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.399881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb01ca-f7a6-4544-a12f-2097d8779112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.400151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb01ca-f7a6-4544-a12f-2097d8779112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.410840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb01ca-f7a6-4544-a12f-2097d8779112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.412297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb01ca-f7a6-4544-a12f-2097d8779112 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.423971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1134/2272] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:55.430249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:55.431725 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789c5aee5f1848989d6ceab773e60007', 'request_id': 'req-21f3106d-4a18-4b25-a355-9ba226f8877a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.442184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789c5aee5f1848989d6ceab773e60007', 'request_id': 'req-21f3106d-4a18-4b25-a355-9ba226f8877a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.448572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789c5aee5f1848989d6ceab773e60007', 'request_id': 'req-21f3106d-4a18-4b25-a355-9ba226f8877a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.460398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] The token cache is being invalidate because project 789c5aee5f1848989d6ceab773e60007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:55.460641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.461396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-21f3106d-4a18-4b25-a355-9ba226f8877a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.463292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1139/2273] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:55 2022] DELETE /identity/v3/projects/789c5aee5f1848989d6ceab773e60007 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:55.469428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce56ed-4029-4f77-b34f-10d9cb431084 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.469736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce56ed-4029-4f77-b34f-10d9cb431084 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.470040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce56ed-4029-4f77-b34f-10d9cb431084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.490847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-33ce56ed-4029-4f77-b34f-10d9cb431084 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:55.492183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-33ce56ed-4029-4f77-b34f-10d9cb431084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:55.557773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1135/2274] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:55.630072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe1a13-0cb0-480e-a2ad-f9a7f88bd418 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.630360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe1a13-0cb0-480e-a2ad-f9a7f88bd418 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.630638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe1a13-0cb0-480e-a2ad-f9a7f88bd418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.659634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-9efe1a13-0cb0-480e-a2ad-f9a7f88bd418 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:55.660930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9efe1a13-0cb0-480e-a2ad-f9a7f88bd418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:55.670608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1140/2275] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:55.674760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce56ed-4029-4f77-b34f-10d9cb431084 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.684865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce56ed-4029-4f77-b34f-10d9cb431084 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.689604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.690210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.690491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.690764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.691416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:55.691753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.694178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.707142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1136/2276] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:55.711321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe1a13-0cb0-480e-a2ad-f9a7f88bd418 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.723066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe1a13-0cb0-480e-a2ad-f9a7f88bd418 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.732841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.733560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.733909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.734249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.735034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.737773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.770046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1141/2277] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:55 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:55.773962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.785263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d61f03-685c-42a3-9122-fc20b85ae84d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.791596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.793786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:55.871765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.872670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.873048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.873647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.874080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.876012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.886670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f95ba-a2e4-4ef0-a158-1e946c6704ef admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.888817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:55.893173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.894116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.894431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.894656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] PATH_INFO: `/v3/projects/aa0dacd264f74bb5a09a8adce00ba20d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.895781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0dacd264f74bb5a09a8adce00ba20d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.897997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.904962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1137/2278] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:55 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:00:55.906554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.916769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e455ed-bdd8-498b-8ed9-8f2a5b2dbb3d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.921552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:55.922144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:55.922430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:55.922699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:55.923340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:55.925001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:55.935909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1138/2279] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:55.939862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.945614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0dacd264f74bb5a09a8adce00ba20d', 'request_id': 'req-09da6fe6-598b-4d81-b111-2d69cc7cbc59'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.957665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0dacd264f74bb5a09a8adce00ba20d', 'request_id': 'req-09da6fe6-598b-4d81-b111-2d69cc7cbc59'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.960629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3cae6-9028-4f71-9fad-7a1ebb721841 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:55.963525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0dacd264f74bb5a09a8adce00ba20d', 'request_id': 'req-09da6fe6-598b-4d81-b111-2d69cc7cbc59'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.976584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] The token cache is being invalidate because project aa0dacd264f74bb5a09a8adce00ba20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:55.976924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.977623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:55.979452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1142/2280] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:55 2022] DELETE /identity/v3/projects/aa0dacd264f74bb5a09a8adce00ba20d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:55.983489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:55.984874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:56.024487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:56.025116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.025392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.025657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.026291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:56.028450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:56.029412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:56.042148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1139/2281] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:00:56.043442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:56.045260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:56.054963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da6fe6-598b-4d81-b111-2d69cc7cbc59 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:56.057489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd2ddf-b463-4148-9596-49b776e43a73 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:56.060534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:56.061186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.061460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.061726 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] PATH_INFO: `/v3/users/91987b28394c491095ae134f516badc2` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.062849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=91987b28394c491095ae134f516badc2)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:56.062959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d0575-4a6c-4bc1-8166-aaf56d207190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:56.063502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d0575-4a6c-4bc1-8166-aaf56d207190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.063784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d0575-4a6c-4bc1-8166-aaf56d207190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.064055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d0575-4a6c-4bc1-8166-aaf56d207190 admin admin] PATH_INFO: `/v3/projects/8990358f1c2e491d94ec2eaf29ea5100/users/9abfe7f3bb204e74a66de99e7a9e8f1f/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.064840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:56.081773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d0575-4a6c-4bc1-8166-aaf56d207190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8990358f1c2e491d94ec2eaf29ea5100, user_id=9abfe7f3bb204e74a66de99e7a9e8f1f, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:56.085078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d0575-4a6c-4bc1-8166-aaf56d207190 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:56.100290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1140/2282] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:56 2022] PUT /identity/v3/projects/8990358f1c2e491d94ec2eaf29ea5100/users/9abfe7f3bb204e74a66de99e7a9e8f1f/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:56.110853 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.111150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.111415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.113449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91987b28394c491095ae134f516badc2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.122252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91987b28394c491095ae134f516badc2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.134855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91987b28394c491095ae134f516badc2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.140631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] MFA Rules not processed for user `9abfe7f3bb204e74a66de99e7a9e8f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:56.141857 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:56.142785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91987b28394c491095ae134f516badc2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.143464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91987b28394c491095ae134f516badc2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.149522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1143/2283] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:00:55 2022] DELETE /identity/v3/users/91987b28394c491095ae134f516badc2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:56.216571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1141/2284] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:00:56.231050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:56.232878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:56.315723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abaa96-dc97-412a-ba05-570e43709c3d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:56.319825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad846e0-f81e-4f6c-a8ec-741eda90375a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022516f130ec4ad6a9b7c46b417a5690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:56.320518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad846e0-f81e-4f6c-a8ec-741eda90375a service glance] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.320910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad846e0-f81e-4f6c-a8ec-741eda90375a service glance] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.321240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad846e0-f81e-4f6c-a8ec-741eda90375a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.403522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad846e0-f81e-4f6c-a8ec-741eda90375a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:56.405019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad846e0-f81e-4f6c-a8ec-741eda90375a service glance] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:56.414818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1144/2285] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:00:56.506220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:56.507733 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:56.564736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:56.582951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1384355c-39f0-4019-afc5-a035669b4079 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:56.641717 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:56.642353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.642629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.642900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] PATH_INFO: `/v3/projects/38d54311285c47f9b5d1ce8d2969f136` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.644121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d54311285c47f9b5d1ce8d2969f136)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:56.646088 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:56.705736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d54311285c47f9b5d1ce8d2969f136', 'request_id': 'req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.713265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d54311285c47f9b5d1ce8d2969f136', 'request_id': 'req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.719756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d54311285c47f9b5d1ce8d2969f136', 'request_id': 'req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.731983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] The token cache is being invalidate because project 38d54311285c47f9b5d1ce8d2969f136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:00:56.732271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.732938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-18ffd8f0-fba3-4d0e-8f53-3ee0e747662d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:00:56.735077 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1142/2286] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:00:56 2022] DELETE /identity/v3/projects/38d54311285c47f9b5d1ce8d2969f136 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:00:56.740061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d010e8-e454-4dad-a90b-544438ee8c43 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.740411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d010e8-e454-4dad-a90b-544438ee8c43 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.740770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d010e8-e454-4dad-a90b-544438ee8c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.763233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-47d010e8-e454-4dad-a90b-544438ee8c43 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:56.764697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-47d010e8-e454-4dad-a90b-544438ee8c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:56.770813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:56.862639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1145/2287] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:56.870233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a41dc3-08d3-4224-b55f-3b589db49373 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.870568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a41dc3-08d3-4224-b55f-3b589db49373 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.870789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a41dc3-08d3-4224-b55f-3b589db49373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.894334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-55a41dc3-08d3-4224-b55f-3b589db49373 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:56.895571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-55a41dc3-08d3-4224-b55f-3b589db49373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:56.906592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1143/2288] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:56.912651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d010e8-e454-4dad-a90b-544438ee8c43 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:56.930613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d010e8-e454-4dad-a90b-544438ee8c43 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:56.936843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:56.937575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.937857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.938340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.939579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:00:56.939579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:56.941215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:56.957449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1146/2289] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:00:56.958640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a41dc3-08d3-4224-b55f-3b589db49373 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:56.969396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a41dc3-08d3-4224-b55f-3b589db49373 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:56.974790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:56.975341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:56.975620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:56.975909 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:56.976545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:56.978515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:57.011728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1144/2290] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:00:56 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:57.013366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:57.026949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976febb-04bc-4363-880c-f745af2920f4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:57.088926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:57.089684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:57.089967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:57.090842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:57.091365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:57.093935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:57.115587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1147/2291] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:00:57 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:00:57.119908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:57.130872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3f488-ed96-45fc-8395-724685972d6c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:57.135644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:57.136241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:57.136522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:57.136908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:57.138196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:57.139931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:57.150461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1145/2292] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:57.155092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:57.166365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b4654-7932-476a-a702-757c52ae5bd7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:57.171889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b32a6e-6d35-45eb-b4ce-c6de993254d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:57.172783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32a6e-6d35-45eb-b4ce-c6de993254d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:57.173108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32a6e-6d35-45eb-b4ce-c6de993254d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:57.173774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b32a6e-6d35-45eb-b4ce-c6de993254d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:57.174073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b32a6e-6d35-45eb-b4ce-c6de993254d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:57.175668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b32a6e-6d35-45eb-b4ce-c6de993254d5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:57.326428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1148/2293] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:00:57.331613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:57.344661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb96e1-029d-4f2f-bde3-28227e311b3f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:57.350290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:57.351176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:57.351760 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:57.352109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] PATH_INFO: `/v3/projects/703bad4a35c14c9394ac02952ca963d0/users/a76595aac0c34a1c94379abdea8c01d2/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:57.373953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] RBAC: Authorizing `identity:create_grant(project_id=703bad4a35c14c9394ac02952ca963d0, user_id=a76595aac0c34a1c94379abdea8c01d2, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:57.376759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:57.407462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1146/2294] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:00:57 2022] PUT /identity/v3/projects/703bad4a35c14c9394ac02952ca963d0/users/a76595aac0c34a1c94379abdea8c01d2/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:00:57.418549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba47cf1-b90d-4ffe-a01c-c4ad41e4528e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:57.418865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba47cf1-b90d-4ffe-a01c-c4ad41e4528e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:57.419179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba47cf1-b90d-4ffe-a01c-c4ad41e4528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:57.461001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7ba47cf1-b90d-4ffe-a01c-c4ad41e4528e None None] MFA Rules not processed for user `a76595aac0c34a1c94379abdea8c01d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:00:57.462434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7ba47cf1-b90d-4ffe-a01c-c4ad41e4528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:57.553826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1149/2295] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:00:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:00:57.570918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:00:57.576159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:00:57.606396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:00:57.648543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518008d-a69f-414f-a874-41bf9ae6529c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:00:57.652306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdbff7-644d-424a-a313-b446550df4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:00:57.653447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdbff7-644d-424a-a313-b446550df4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:00:57.653800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdbff7-644d-424a-a313-b446550df4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:00:57.654138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdbff7-644d-424a-a313-b446550df4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:00:57.662953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdbff7-644d-424a-a313-b446550df4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:00:57.664402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdbff7-644d-424a-a313-b446550df4b6 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:00:57.674180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1147/2296] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:00:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:02.652677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba47cf1-b90d-4ffe-a01c-c4ad41e4528e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:02.670758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:02.791567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba47cf1-b90d-4ffe-a01c-c4ad41e4528e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:02.798768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297080-6223-4046-a294-187eadee767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:02.799889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297080-6223-4046-a294-187eadee767c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:02.800103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297080-6223-4046-a294-187eadee767c admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:02.800329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08297080-6223-4046-a294-187eadee767c admin admin] PATH_INFO: `/v3/users/a76595aac0c34a1c94379abdea8c01d2` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:02.802723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297080-6223-4046-a294-187eadee767c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76595aac0c34a1c94379abdea8c01d2)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:02.804961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08297080-6223-4046-a294-187eadee767c admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:02.912183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-08297080-6223-4046-a294-187eadee767c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76595aac0c34a1c94379abdea8c01d2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:02.922741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-08297080-6223-4046-a294-187eadee767c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76595aac0c34a1c94379abdea8c01d2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:02.944536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-08297080-6223-4046-a294-187eadee767c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76595aac0c34a1c94379abdea8c01d2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:02.954087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-08297080-6223-4046-a294-187eadee767c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76595aac0c34a1c94379abdea8c01d2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:02.954897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-08297080-6223-4046-a294-187eadee767c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76595aac0c34a1c94379abdea8c01d2'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:02.969032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1150/2297] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:01:02 2022] DELETE /identity/v3/users/a76595aac0c34a1c94379abdea8c01d2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:02.971648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:02.972152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:02.972152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:03.005937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:03.007223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:03.203982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1148/2298] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:03.220985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297080-6223-4046-a294-187eadee767c admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:03.223136 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-08297080-6223-4046-a294-187eadee767c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:03.331016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08297080-6223-4046-a294-187eadee767c admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:03.338861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b0e6d-a541-49db-81df-9bafdaab4dc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:03.339494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0e6d-a541-49db-81df-9bafdaab4dc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:03.339771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0e6d-a541-49db-81df-9bafdaab4dc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:03.340047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b0e6d-a541-49db-81df-9bafdaab4dc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:03.352087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b0e6d-a541-49db-81df-9bafdaab4dc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:03.354033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b0e6d-a541-49db-81df-9bafdaab4dc5 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:03.367991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1151/2299] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:03.757430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:03.759241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:03.786283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6645e-89d6-437b-944e-320b34b5c20a None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:03.809513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:03.810343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:03.810626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:03.810899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] PATH_INFO: `/v3/projects/703bad4a35c14c9394ac02952ca963d0` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:03.813693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] RBAC: Authorizing `identity:delete_project(project_id=703bad4a35c14c9394ac02952ca963d0)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:03.815635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:03.880758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703bad4a35c14c9394ac02952ca963d0', 'request_id': 'req-10585494-82ab-4e8f-95c3-d8f8bd35135a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:03.890290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703bad4a35c14c9394ac02952ca963d0', 'request_id': 'req-10585494-82ab-4e8f-95c3-d8f8bd35135a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:03.896226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703bad4a35c14c9394ac02952ca963d0', 'request_id': 'req-10585494-82ab-4e8f-95c3-d8f8bd35135a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:03.908766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] The token cache is being invalidate because project 703bad4a35c14c9394ac02952ca963d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:01:03.909004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:03.909647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-10585494-82ab-4e8f-95c3-d8f8bd35135a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:03.911144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1149/2300] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:01:03 2022] DELETE /identity/v3/projects/703bad4a35c14c9394ac02952ca963d0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:03.916699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506546b-9640-4b12-b0bd-fd71f590e92e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:03.917323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506546b-9640-4b12-b0bd-fd71f590e92e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:03.917323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506546b-9640-4b12-b0bd-fd71f590e92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:03.937423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7506546b-9640-4b12-b0bd-fd71f590e92e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:03.939130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7506546b-9640-4b12-b0bd-fd71f590e92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:03.972557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:04.011492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1152/2301] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:01:04.020359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7e4b-40d7-429d-a9bd-0c0134307925 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.020949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7e4b-40d7-429d-a9bd-0c0134307925 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.021060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7e4b-40d7-429d-a9bd-0c0134307925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.042860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-92ba7e4b-40d7-429d-a9bd-0c0134307925 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:04.044289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-92ba7e4b-40d7-429d-a9bd-0c0134307925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:04.052964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1150/2302] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:01:04.057243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506546b-9640-4b12-b0bd-fd71f590e92e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:04.066939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506546b-9640-4b12-b0bd-fd71f590e92e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:04.071772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:04.072369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.072646 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.072952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.073614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:01:04.073917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:04.075514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:04.087881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1153/2303] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:01:04.093014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7e4b-40d7-429d-a9bd-0c0134307925 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:04.102457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7e4b-40d7-429d-a9bd-0c0134307925 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:04.107239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:04.107856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.108157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.108426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.109120 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:04.111046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:04.139724 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1151/2304] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:01:04 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:01:04.144346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:04.155877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92060a94-da68-438a-b0e4-fd03f50f6e52 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:04.215204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:04.215788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.216061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.216333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.218020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:04.219946 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:04.245472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1154/2305] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:01:04 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:01:04.250523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:04.265783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18048deb-9a2b-4f09-8415-d39fd32eecdd admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:04.270773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:04.271462 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.271816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.272210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.272968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:04.274654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:04.286445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1152/2306] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:01:04.290744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:04.304678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ea2d2-8bda-4c52-852f-e566721b16e9 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:04.309565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddb238-c6ea-4049-99ce-7739564785d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:04.310881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb238-c6ea-4049-99ce-7739564785d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.311369 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb238-c6ea-4049-99ce-7739564785d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.311647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb238-c6ea-4049-99ce-7739564785d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.312406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddb238-c6ea-4049-99ce-7739564785d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:04.319870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddb238-c6ea-4049-99ce-7739564785d5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:04.335855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1155/2307] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:01:04.337664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:04.353484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e9077-a9eb-4962-af8a-f07508d141d8 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:04.359849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:04.360703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.361012 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.361424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] PATH_INFO: `/v3/projects/df40819799cf4e0cb9f8bb69ee6e6813/users/87f51f11bb92407fa0cac3b5702d84bc/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.367693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:04.386762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df40819799cf4e0cb9f8bb69ee6e6813, user_id=87f51f11bb92407fa0cac3b5702d84bc, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:04.391692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:04.412759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1153/2308] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:01:04 2022] PUT /identity/v3/projects/df40819799cf4e0cb9f8bb69ee6e6813/users/87f51f11bb92407fa0cac3b5702d84bc/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:04.422613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244ebaa-3727-4cd1-bb9e-ef55f520308c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.422906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244ebaa-3727-4cd1-bb9e-ef55f520308c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.423220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244ebaa-3727-4cd1-bb9e-ef55f520308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.465811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-1244ebaa-3727-4cd1-bb9e-ef55f520308c None None] MFA Rules not processed for user `87f51f11bb92407fa0cac3b5702d84bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:04.469329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1244ebaa-3727-4cd1-bb9e-ef55f520308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:04.590598 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1156/2309] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:04.602321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:04.604534 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:04.682471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4eebd-0557-4743-ac72-32e747022105 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:04.687171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:04.687780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:04.688054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:04.688322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:04.699179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:04.700665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:04.712818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1154/2310] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:10.854693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:10.855206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:10.855301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:10.877346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:10.878877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:10.899054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:10.950336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1157/2311] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:01:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:11.049334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:11.052509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:11.149070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8a6da-1004-4262-9cf1-f14d1f9608de service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:11.162697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba12f9-7245-4b75-a46a-dd67c0c5d826 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:11.163478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba12f9-7245-4b75-a46a-dd67c0c5d826 service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:11.164511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba12f9-7245-4b75-a46a-dd67c0c5d826 service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:11.165016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba12f9-7245-4b75-a46a-dd67c0c5d826 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:11.177533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba12f9-7245-4b75-a46a-dd67c0c5d826 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:11.180787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba12f9-7245-4b75-a46a-dd67c0c5d826 service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:11.190925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1155/2312] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:01:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:01:11.511760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:11.513503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:11.612407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaef05-e827-44db-afb1-0fc989296589 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:11.621225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917de032-1463-488f-b732-229fe098b105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:11.622219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de032-1463-488f-b732-229fe098b105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:11.623888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de032-1463-488f-b732-229fe098b105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:11.624279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917de032-1463-488f-b732-229fe098b105 admin admin] PATH_INFO: `/v3/users/0f5317cfdeb6462bbb2091fb99e54597` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:11.625821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917de032-1463-488f-b732-229fe098b105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5317cfdeb6462bbb2091fb99e54597)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:11.628055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917de032-1463-488f-b732-229fe098b105 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:11.696007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5317cfdeb6462bbb2091fb99e54597'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:11.704929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5317cfdeb6462bbb2091fb99e54597'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:11.743816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5317cfdeb6462bbb2091fb99e54597'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:11.759617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5317cfdeb6462bbb2091fb99e54597'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:11.760381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5317cfdeb6462bbb2091fb99e54597'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:11.767437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1158/2313] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:01:11 2022] DELETE /identity/v3/users/0f5317cfdeb6462bbb2091fb99e54597 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:11.772138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:11.772440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:11.772707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:11.798465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:11.799828 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:11.896763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1156/2314] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:11.906315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:11.909211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:11.999421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:12.005620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917de032-1463-488f-b732-229fe098b105 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:12.015409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaab17-1467-40ae-8f07-bc544a980f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:12.016403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaab17-1467-40ae-8f07-bc544a980f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:12.016998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaab17-1467-40ae-8f07-bc544a980f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:12.017327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaab17-1467-40ae-8f07-bc544a980f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:12.029405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaab17-1467-40ae-8f07-bc544a980f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:12.030870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaab17-1467-40ae-8f07-bc544a980f5f service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:12.041344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1159/2315] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:12.651742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:12.653313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:12.676936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78d6e-f934-4d18-84a6-653a3b8d9fc1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:12.689095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:12.689859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:12.690376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:12.690988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] PATH_INFO: `/v3/projects/afe649b53db84533af6093e7f7207c18` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:12.693163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe649b53db84533af6093e7f7207c18)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:12.695346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:12.731887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:12.795469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe649b53db84533af6093e7f7207c18', 'request_id': 'req-419fed06-a021-4f00-83d0-7a9d7c602ddb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:12.804453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe649b53db84533af6093e7f7207c18', 'request_id': 'req-419fed06-a021-4f00-83d0-7a9d7c602ddb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:12.812855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe649b53db84533af6093e7f7207c18', 'request_id': 'req-419fed06-a021-4f00-83d0-7a9d7c602ddb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:12.826192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] The token cache is being invalidate because project afe649b53db84533af6093e7f7207c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:01:12.826739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:12.827461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-419fed06-a021-4f00-83d0-7a9d7c602ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:12.829618 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1157/2316] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:01:12 2022] DELETE /identity/v3/projects/afe649b53db84533af6093e7f7207c18 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:12.834380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4f063-24f9-418c-9043-98203a59bce8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:12.834610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4f063-24f9-418c-9043-98203a59bce8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:12.834883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4f063-24f9-418c-9043-98203a59bce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:12.875917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c1f4f063-24f9-418c-9043-98203a59bce8 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:12.877112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c1f4f063-24f9-418c-9043-98203a59bce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:12.950075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1160/2317] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:01:12.959372 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d081c98-9dd4-438f-9ad8-b394711d4ab5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:12.959685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d081c98-9dd4-438f-9ad8-b394711d4ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:12.959975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d081c98-9dd4-438f-9ad8-b394711d4ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:12.994719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-8d081c98-9dd4-438f-9ad8-b394711d4ab5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:12.996105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8d081c98-9dd4-438f-9ad8-b394711d4ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:13.013217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1158/2318] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:13.017810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4f063-24f9-418c-9043-98203a59bce8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:13.031232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4f063-24f9-418c-9043-98203a59bce8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:13.036736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:13.037600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.038077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.039153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.040013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:01:13.040485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:13.042325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:13.058083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1161/2319] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:01:13.063118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d081c98-9dd4-438f-9ad8-b394711d4ab5 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:13.094643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d081c98-9dd4-438f-9ad8-b394711d4ab5 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:13.104387 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:13.105666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.105951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.106227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.107504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:13.110237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:13.194465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1159/2320] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:01:13 2022] POST /identity/v3/projects => generated 397 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:01:13.199513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:13.213569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241b44e-550f-4766-b45c-cb9af8ca3744 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:13.290213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:13.291125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.291401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.291674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.292316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:13.295046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:13.329488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1162/2321] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:01:13 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:01:13.333744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:13.346642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29588140-c7c2-48ec-a409-8f7e1ae88e59 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:13.352416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:13.353170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.353522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.353898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.354601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:13.356332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:13.374260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1160/2322] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:01:13.377127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:13.391427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d2fad-6bb8-441a-8242-c017865dd271 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:13.396632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897373d9-1df3-4eff-b82e-855a452b6f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:13.397602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897373d9-1df3-4eff-b82e-855a452b6f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.397867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897373d9-1df3-4eff-b82e-855a452b6f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.398146 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897373d9-1df3-4eff-b82e-855a452b6f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.398794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897373d9-1df3-4eff-b82e-855a452b6f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:13.400402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897373d9-1df3-4eff-b82e-855a452b6f76 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:13.414063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1163/2323] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:01:13.418675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:13.431236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aed390-0ede-4362-a316-ced7a6236bb1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:13.436567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:13.437575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.438010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.438336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] PATH_INFO: `/v3/projects/6720180fe1ec4adb9c485507333e9140/users/9e72f5ddf26441dd9f5af45535332a20/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.455850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6720180fe1ec4adb9c485507333e9140, user_id=9e72f5ddf26441dd9f5af45535332a20, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:13.459431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:13.477229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1161/2324] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:01:13 2022] PUT /identity/v3/projects/6720180fe1ec4adb9c485507333e9140/users/9e72f5ddf26441dd9f5af45535332a20/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:01:13.489470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.489751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.490020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.523408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] MFA Rules not processed for user `9e72f5ddf26441dd9f5af45535332a20`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:13.524563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:13.579511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:13.605534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1164/2325] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:13.619713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:13.636400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:13.733391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8becd4-c3a4-4771-838e-036cd2ef4d54 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:13.737423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:13.738194 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:13.738602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:13.738976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:13.752435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:13.754087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:13.766472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1162/2326] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:20.687984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:20.689374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:20.799890 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695da36d-2354-492a-9d40-3cdc1612a2a0 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:20.804059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c156210-b387-4dd7-930d-ac3318e36175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:20.804679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c156210-b387-4dd7-930d-ac3318e36175 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:20.804948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c156210-b387-4dd7-930d-ac3318e36175 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:20.805214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c156210-b387-4dd7-930d-ac3318e36175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:20.807006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2c156210-b387-4dd7-930d-ac3318e36175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:20.899055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c156210-b387-4dd7-930d-ac3318e36175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:20.900740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c156210-b387-4dd7-930d-ac3318e36175 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:20.916904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1165/2327] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:23.761420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:25.726525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:25.808078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58817a-ae57-4848-85af-74626ce975c1 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:25.814247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:25.814899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:25.815215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:25.815568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] PATH_INFO: `/v3/users/9e72f5ddf26441dd9f5af45535332a20` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:25.816585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e72f5ddf26441dd9f5af45535332a20)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:25.818579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:25.899365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e72f5ddf26441dd9f5af45535332a20'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:25.900581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e72f5ddf26441dd9f5af45535332a20'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:25.912908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e72f5ddf26441dd9f5af45535332a20'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:25.935784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e72f5ddf26441dd9f5af45535332a20'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:25.946666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e72f5ddf26441dd9f5af45535332a20'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:25.956577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1163/2328] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:01:25 2022] DELETE /identity/v3/users/9e72f5ddf26441dd9f5af45535332a20 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:25.959149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:25.960031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:25.960300 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:25.987874 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:25.990018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:26.084350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1166/2329] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:26.091625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:26.093226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:26.178261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b3c38-00b3-4b15-ab61-209403b27a42 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:26.182431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db30d2d-107a-413c-92f9-dcc077c06beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:26.183085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db30d2d-107a-413c-92f9-dcc077c06beb service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:26.183568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db30d2d-107a-413c-92f9-dcc077c06beb service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:26.183867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db30d2d-107a-413c-92f9-dcc077c06beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:26.196811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db30d2d-107a-413c-92f9-dcc077c06beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:26.198714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db30d2d-107a-413c-92f9-dcc077c06beb service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:26.214790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1164/2330] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:26.596017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:26.598462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:26.626983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b8651-5349-4d00-bfda-64568dea7d4b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:26.634633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:26.635952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:26.636239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:26.636755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] PATH_INFO: `/v3/projects/6720180fe1ec4adb9c485507333e9140` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:26.638951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6720180fe1ec4adb9c485507333e9140)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:26.642225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:26.696927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:26.711571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6720180fe1ec4adb9c485507333e9140', 'request_id': 'req-62ebce95-d925-4d1c-a895-b3c3d9de1f45'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:26.724871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6720180fe1ec4adb9c485507333e9140', 'request_id': 'req-62ebce95-d925-4d1c-a895-b3c3d9de1f45'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:26.732620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6720180fe1ec4adb9c485507333e9140', 'request_id': 'req-62ebce95-d925-4d1c-a895-b3c3d9de1f45'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:26.745409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] The token cache is being invalidate because project 6720180fe1ec4adb9c485507333e9140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:01:26.745846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:26.746639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-62ebce95-d925-4d1c-a895-b3c3d9de1f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:26.752672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1167/2331] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:01:26 2022] DELETE /identity/v3/projects/6720180fe1ec4adb9c485507333e9140 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:26.754215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1d2e5-edaf-4da9-a312-7865bfa41568 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:26.755153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1d2e5-edaf-4da9-a312-7865bfa41568 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:26.755436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1d2e5-edaf-4da9-a312-7865bfa41568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:26.787949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-fef1d2e5-edaf-4da9-a312-7865bfa41568 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:26.790898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fef1d2e5-edaf-4da9-a312-7865bfa41568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:26.822409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:26.900172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1165/2332] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:01:26.916224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21319ebd-2bbe-466f-8484-e4736c6acf94 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:26.916684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21319ebd-2bbe-466f-8484-e4736c6acf94 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:26.917253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21319ebd-2bbe-466f-8484-e4736c6acf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:26.975081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-21319ebd-2bbe-466f-8484-e4736c6acf94 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:26.976417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-21319ebd-2bbe-466f-8484-e4736c6acf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:26.985287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1168/2333] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:26.992640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1d2e5-edaf-4da9-a312-7865bfa41568 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:27.004685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1d2e5-edaf-4da9-a312-7865bfa41568 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:27.011500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:27.012155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.013017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.013294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.014063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:01:27.014371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:27.016082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:27.054624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1166/2334] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:01:27.055422 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21319ebd-2bbe-466f-8484-e4736c6acf94 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:27.075964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21319ebd-2bbe-466f-8484-e4736c6acf94 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:27.088218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:27.089160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.089432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.089693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.090427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:27.107096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:27.158751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1169/2335] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:01:27 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:01:27.178300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:27.211109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb598cd-40db-42c6-bd11-26e0cd6ea0dd admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:27.311391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:27.312394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.312873 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.313309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.314004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:27.316060 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:27.343980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1167/2336] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:01:27 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:01:27.345374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:27.355707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdffc96-af50-4bab-aad1-93cdeff2bf65 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:27.361328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:27.362008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.362346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.362695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.363404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:27.366139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:27.378332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1170/2337] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:01:27.381400 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:27.397679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657917c-ab5f-43d1-a6f2-3129fa70a22b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:27.405856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b324d-314c-4294-896e-75d54243f7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:27.406638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b324d-314c-4294-896e-75d54243f7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.406948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b324d-314c-4294-896e-75d54243f7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.407220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b324d-314c-4294-896e-75d54243f7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.409041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b324d-314c-4294-896e-75d54243f7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:27.411833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b324d-314c-4294-896e-75d54243f7e5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:27.433248 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1168/2338] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:01:27.436159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:27.452143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fdbc9-578b-4414-beaf-e3a6a70f0ad6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:27.459670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:27.460426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.461511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.461926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] PATH_INFO: `/v3/projects/40732e98240b4797b941e57f6606e6c5/users/7377a1bca391446eb36b2a3233a7d471/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.483338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40732e98240b4797b941e57f6606e6c5, user_id=7377a1bca391446eb36b2a3233a7d471, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:27.486054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:27.506677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1171/2339] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:01:27 2022] PUT /identity/v3/projects/40732e98240b4797b941e57f6606e6c5/users/7377a1bca391446eb36b2a3233a7d471/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:27.509269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99152a65-b217-49ac-8ead-041893f36039 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.509632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99152a65-b217-49ac-8ead-041893f36039 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.509973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99152a65-b217-49ac-8ead-041893f36039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.533966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-99152a65-b217-49ac-8ead-041893f36039 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:27.535290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-99152a65-b217-49ac-8ead-041893f36039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:27.645277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1169/2340] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:27.653931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:27.656155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:27.745301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d1f42-726f-458d-9308-f6a0c70a586a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:27.750590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:27.751295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:27.751626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:27.751972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:27.766210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:27.767747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:27.779114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1172/2341] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:33.406682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb8fbb-c94e-40d8-bdb5-1ec7fb439cf9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:33.407230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb8fbb-c94e-40d8-bdb5-1ec7fb439cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:33.407230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb8fbb-c94e-40d8-bdb5-1ec7fb439cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:33.455702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c8cb8fbb-c94e-40d8-bdb5-1ec7fb439cf9 None None] MFA Rules not processed for user `7377a1bca391446eb36b2a3233a7d471`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:33.457124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c8cb8fbb-c94e-40d8-bdb5-1ec7fb439cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:33.530071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:33.726892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1170/2342] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:33.728187 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:33.745411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad150e-6ce1-4ab3-9f51-c738e918c394 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:33.766343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:33.767105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:33.767477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:33.767948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:33.769311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:33.775333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:33.846010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1173/2343] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:01:33 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:01:33.850912 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb8fbb-c94e-40d8-bdb5-1ec7fb439cf9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:33.864176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb8fbb-c94e-40d8-bdb5-1ec7fb439cf9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:33.972487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:33.973197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:33.973482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:33.973755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:33.974446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:33.976444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:34.008882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1171/2344] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:01:33 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:01:34.010855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:34.026536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737a15f-800f-44a3-9193-bccfbc2c4273 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:34.049476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:34.050551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:34.050843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:34.051119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:34.051785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:34.053438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:34.079649 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1174/2345] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:01:34.083646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:34.099779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61daf478-0497-490c-b72b-6e06f0fe613e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:34.122139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:34.122848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:34.123127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:34.123395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] PATH_INFO: `/v3/projects/cc473a71236e4ccf9e304f1e1e309c81/users/4dbcb844983a46ddbf28a1ca12e3414c/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:34.148919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc473a71236e4ccf9e304f1e1e309c81, user_id=4dbcb844983a46ddbf28a1ca12e3414c, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:34.151771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:34.188764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1172/2346] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:01:34 2022] PUT /identity/v3/projects/cc473a71236e4ccf9e304f1e1e309c81/users/4dbcb844983a46ddbf28a1ca12e3414c/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:34.192450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:34.209114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a89907-7eb0-4101-9afd-3d34f286b959 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:34.275553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:34.390274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1b9f9-2cf7-4e64-981c-ee8ded3109f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:34.391128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1b9f9-2cf7-4e64-981c-ee8ded3109f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:34.391409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1b9f9-2cf7-4e64-981c-ee8ded3109f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:34.391855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1b9f9-2cf7-4e64-981c-ee8ded3109f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:34.392520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1b9f9-2cf7-4e64-981c-ee8ded3109f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:34.395195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1b9f9-2cf7-4e64-981c-ee8ded3109f9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:34.410633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1175/2347] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:01:34.414601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:34.447810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51f6e1-7b42-486f-8f73-4c2f659aa1c4 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:34.467798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:34.468481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:34.468818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:34.469093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] PATH_INFO: `/v3/projects/cc473a71236e4ccf9e304f1e1e309c81/users/4dbcb844983a46ddbf28a1ca12e3414c/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:34.472779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc473a71236e4ccf9e304f1e1e309c81, user_id=4dbcb844983a46ddbf28a1ca12e3414c, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:34.474559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:34.495385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1173/2348] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:01:34 2022] PUT /identity/v3/projects/cc473a71236e4ccf9e304f1e1e309c81/users/4dbcb844983a46ddbf28a1ca12e3414c/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:37.282006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1089c-6779-4f63-a559-9b3b42de4f87 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:37.282957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1089c-6779-4f63-a559-9b3b42de4f87 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:37.283407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1089c-6779-4f63-a559-9b3b42de4f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:37.323301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-11e1089c-6779-4f63-a559-9b3b42de4f87 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:37.330800 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-11e1089c-6779-4f63-a559-9b3b42de4f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:37.491137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1176/2349] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 16:01:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:37.501967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:37.504365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:37.732781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868db6c9-4e85-4f8a-a322-0570970c86f3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:37.739053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:37.739876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:37.740326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:37.740890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:37.760826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:37.762762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:37.770708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:37.783259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1174/2350] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:01:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:01:41.065711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:41.066248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:41.066435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:41.101232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] MFA Rules not processed for user `4dbcb844983a46ddbf28a1ca12e3414c`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:41.102526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:41.181643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1177/2351] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:41.195328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:41.196842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:41.271218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe862c5-d97a-4cac-ba87-8b4f0770f123 service swift] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:41.276648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7afcd-1e53-417e-aece-9eac604720f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:41.277854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7afcd-1e53-417e-aece-9eac604720f5 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:41.278129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7afcd-1e53-417e-aece-9eac604720f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:41.278396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7afcd-1e53-417e-aece-9eac604720f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:41.351055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7afcd-1e53-417e-aece-9eac604720f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:41.352696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7afcd-1e53-417e-aece-9eac604720f5 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:41.363645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1175/2352] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:43.725264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:43.727861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:43.820679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:43.837446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad77a6b-fbb2-4431-8e81-cac04d4e0fca None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:43.844027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:43.844670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:43.844980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:43.845253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] PATH_INFO: `/v3/users/eeb6f39c8c2b4a2f9ae77a20a20b655c` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:43.847871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb6f39c8c2b4a2f9ae77a20a20b655c)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:43.849868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:43.945201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb6f39c8c2b4a2f9ae77a20a20b655c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:43.945629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbcc16-db1b-4638-bebc-cb7c575d74f1 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:43.946506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbcc16-db1b-4638-bebc-cb7c575d74f1 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:43.947000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbcc16-db1b-4638-bebc-cb7c575d74f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:43.959422 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb6f39c8c2b4a2f9ae77a20a20b655c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:43.997835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb6f39c8c2b4a2f9ae77a20a20b655c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:43.999696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-0efbcc16-db1b-4638-bebc-cb7c575d74f1 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:44.005009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0efbcc16-db1b-4638-bebc-cb7c575d74f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:44.009410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb6f39c8c2b4a2f9ae77a20a20b655c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:44.010294 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb6f39c8c2b4a2f9ae77a20a20b655c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:44.015601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1176/2353] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:01:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:44.018457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1178/2354] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:01:43 2022] DELETE /identity/v3/users/eeb6f39c8c2b4a2f9ae77a20a20b655c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:44.030597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbcc16-db1b-4638-bebc-cb7c575d74f1 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:44.045301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbcc16-db1b-4638-bebc-cb7c575d74f1 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:44.133714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ef3c5-f11f-4d5a-bf50-fa72f66a9952 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:44.134628 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ef3c5-f11f-4d5a-bf50-fa72f66a9952 service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:44.135492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ef3c5-f11f-4d5a-bf50-fa72f66a9952 service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:44.136016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ef3c5-f11f-4d5a-bf50-fa72f66a9952 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:44.159007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3a9ef3c5-f11f-4d5a-bf50-fa72f66a9952 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:44.228191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:44.263946 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:44.265810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:44.275655 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ef3c5-f11f-4d5a-bf50-fa72f66a9952 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:44.284444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ef3c5-f11f-4d5a-bf50-fa72f66a9952 service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:44.295587 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1177/2355] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:01:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:01:44.393942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0248bcc-3b9f-42a8-8985-ddb1d9516be2 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:44.411814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:44.413059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:44.413708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:44.414172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] PATH_INFO: `/v3/projects/1616ab7beef7499280c47114208878fe` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:44.420418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1616ab7beef7499280c47114208878fe)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:44.423744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:44.485340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1616ab7beef7499280c47114208878fe', 'request_id': 'req-25866d95-933a-4391-9c78-a21573e7461e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:44.497851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1616ab7beef7499280c47114208878fe', 'request_id': 'req-25866d95-933a-4391-9c78-a21573e7461e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:44.505461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1616ab7beef7499280c47114208878fe', 'request_id': 'req-25866d95-933a-4391-9c78-a21573e7461e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:44.518042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] The token cache is being invalidate because project 1616ab7beef7499280c47114208878fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:01:44.518341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:44.519019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-25866d95-933a-4391-9c78-a21573e7461e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:44.520940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1179/2356] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:01:44 2022] DELETE /identity/v3/projects/1616ab7beef7499280c47114208878fe => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:01:44.525969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefd58-6e85-4767-aac9-4dd188977adb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:44.526264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefd58-6e85-4767-aac9-4dd188977adb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:44.526553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aefd58-6e85-4767-aac9-4dd188977adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:44.599230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-08aefd58-6e85-4767-aac9-4dd188977adb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:44.602952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-08aefd58-6e85-4767-aac9-4dd188977adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:44.880709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ed38-01cb-498b-b3b3-d6ef1c9c03c1 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:44.881250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ed38-01cb-498b-b3b3-d6ef1c9c03c1 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:44.881404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640ed38-01cb-498b-b3b3-d6ef1c9c03c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:44.885518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1178/2357] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:44.896361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:44.921384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-5640ed38-01cb-498b-b3b3-d6ef1c9c03c1 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:44.923808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5640ed38-01cb-498b-b3b3-d6ef1c9c03c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:44.935749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1180/2358] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:44.940374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefd58-6e85-4767-aac9-4dd188977adb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:44.955790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aefd58-6e85-4767-aac9-4dd188977adb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:44.962755 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:44.963815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:44.964263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:44.965257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:44.966381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:01:44.966956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:44.968776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:45.005010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1179/2359] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:01:45.009993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640ed38-01cb-498b-b3b3-d6ef1c9c03c1 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:45.029246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640ed38-01cb-498b-b3b3-d6ef1c9c03c1 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:45.040116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:45.040981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:45.042035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:45.042327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:45.044564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:45.048480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:45.106106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1181/2360] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:01:45 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:01:45.110308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:45.126432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f15f3-a6ec-484c-8e8e-36829fc46caf admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:45.210855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:45.211682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:45.212199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:45.212479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:45.213290 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:45.216621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:45.259173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1180/2361] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:01:45 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:01:45.263325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:45.276165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f45fc-ba91-4074-9bbe-c340402dacca admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:45.282816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:45.283446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:45.283728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:45.284001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:45.284662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:45.286296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:45.328015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1182/2362] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:01:45.331947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:45.347211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f1093-0744-4c50-bf49-632d67cbdad2 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:45.364938 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6314ea-5b42-4574-824e-ce9f17e7437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:45.365759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6314ea-5b42-4574-824e-ce9f17e7437b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:45.366106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6314ea-5b42-4574-824e-ce9f17e7437b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:45.366468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6314ea-5b42-4574-824e-ce9f17e7437b admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:45.367230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6314ea-5b42-4574-824e-ce9f17e7437b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:45.369511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6314ea-5b42-4574-824e-ce9f17e7437b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:45.384951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1181/2363] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:01:45.386164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:45.401754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc994dc-5561-47ca-8792-c7ce9274cd0d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:45.414190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:45.415184 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:45.415820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:45.416113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] PATH_INFO: `/v3/projects/f8c41c14b0b3411d9b7e39e3983f2ccd/users/5d59791fcdee41149fce9b8a950963fc/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:45.446255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c41c14b0b3411d9b7e39e3983f2ccd, user_id=5d59791fcdee41149fce9b8a950963fc, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:45.449809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:45.483026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1183/2364] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:01:45 2022] PUT /identity/v3/projects/f8c41c14b0b3411d9b7e39e3983f2ccd/users/5d59791fcdee41149fce9b8a950963fc/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:45.485043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358721-7b5c-49e5-89b8-8ad7d83ccb76 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:45.485331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358721-7b5c-49e5-89b8-8ad7d83ccb76 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:45.485605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358721-7b5c-49e5-89b8-8ad7d83ccb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:45.523897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-10358721-7b5c-49e5-89b8-8ad7d83ccb76 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:45.525065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-10358721-7b5c-49e5-89b8-8ad7d83ccb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:45.607270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:45.646477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1182/2365] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:45.658057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:45.660843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:45.763722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612066b-a7c2-4ea5-87ec-8ed526bd8136 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:45.771503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:45.772346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:45.772625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:45.773112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:45.801189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:45.802592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:45.827323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1184/2366] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:47.204072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:47.204544 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:47.204642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:47.304302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] MFA Rules not processed for user `5d59791fcdee41149fce9b8a950963fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:47.311265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:47.562338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1183/2367] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:47.575138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:47.577838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:47.784976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a357c-528f-4f15-812b-c536bc846fc7 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:47.791182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f9c81-bd9e-4250-9bc7-fdfab043d885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:47.792168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9c81-bd9e-4250-9bc7-fdfab043d885 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:47.793342 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9c81-bd9e-4250-9bc7-fdfab043d885 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:47.793966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f9c81-bd9e-4250-9bc7-fdfab043d885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:47.816062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f9c81-bd9e-4250-9bc7-fdfab043d885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:47.818574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f9c81-bd9e-4250-9bc7-fdfab043d885 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:47.841541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1185/2368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:48.532647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:48.535265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:48.556357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec67fe5-a14d-4cd4-911c-4fe92fd6c87f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:48.565889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:48.566983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:48.567958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:48.568401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] PATH_INFO: `/v3/users/9abfe7f3bb204e74a66de99e7a9e8f1f` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:48.570016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abfe7f3bb204e74a66de99e7a9e8f1f)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:48.575827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:48.736253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abfe7f3bb204e74a66de99e7a9e8f1f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:48.737682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abfe7f3bb204e74a66de99e7a9e8f1f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:48.758906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abfe7f3bb204e74a66de99e7a9e8f1f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:48.786707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abfe7f3bb204e74a66de99e7a9e8f1f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:48.835466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abfe7f3bb204e74a66de99e7a9e8f1f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:48.863217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1184/2369] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:01:48 2022] DELETE /identity/v3/users/9abfe7f3bb204e74a66de99e7a9e8f1f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:48.867671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bcfc7-49ad-420f-8422-7cbb87f0a77a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:48.868750 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bcfc7-49ad-420f-8422-7cbb87f0a77a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:48.869463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bcfc7-49ad-420f-8422-7cbb87f0a77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:48.933919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d97bcfc7-49ad-420f-8422-7cbb87f0a77a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:48.942153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d97bcfc7-49ad-420f-8422-7cbb87f0a77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:49.053647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:49.083178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1186/2370] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:49.090691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:49.093295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:49.230159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:01:49.294621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79139b3d-2a67-4a2c-a8f6-ee1f077c1759 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:49.305856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:49.306254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:49.306547 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:49.307110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:49.339152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:49.341637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:49.356155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1185/2371] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:01:49.560159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aacbf3-f7ac-42e6-9837-ab274ecbe4b5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:49.560515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aacbf3-f7ac-42e6-9837-ab274ecbe4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:49.560698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aacbf3-f7ac-42e6-9837-ab274ecbe4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:49.609386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-06aacbf3-f7ac-42e6-9837-ab274ecbe4b5 None None] MFA Rules not processed for user `5d59791fcdee41149fce9b8a950963fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:49.611088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-06aacbf3-f7ac-42e6-9837-ab274ecbe4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:49.739046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1187/2372] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:49.782193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:01:49.785389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:49.812026 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a63c-9e8c-49d2-bf8a-a507fe50a292 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:01:49.830947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:01:49.831605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:49.832064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:49.837310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] PATH_INFO: `/v3/projects/8990358f1c2e491d94ec2eaf29ea5100` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:49.844550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8990358f1c2e491d94ec2eaf29ea5100)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:01:49.846943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:01:49.907505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8990358f1c2e491d94ec2eaf29ea5100', 'request_id': 'req-33384f36-bfd3-417a-9425-cf1d46e6f10e'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:49.923981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8990358f1c2e491d94ec2eaf29ea5100', 'request_id': 'req-33384f36-bfd3-417a-9425-cf1d46e6f10e'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:49.935894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8990358f1c2e491d94ec2eaf29ea5100', 'request_id': 'req-33384f36-bfd3-417a-9425-cf1d46e6f10e'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:49.973434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] The token cache is being invalidate because project 8990358f1c2e491d94ec2eaf29ea5100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:01:49.973901 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:49.974699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-33384f36-bfd3-417a-9425-cf1d46e6f10e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:01:49.976578 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1186/2373] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:01:49 2022] DELETE /identity/v3/projects/8990358f1c2e491d94ec2eaf29ea5100 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:01:54.846686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f7ce-fb40-49c5-bf2d-9468488baad2 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:01:54.847245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f7ce-fb40-49c5-bf2d-9468488baad2 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:01:54.847640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97f7ce-fb40-49c5-bf2d-9468488baad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:01:54.890076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-5b97f7ce-fb40-49c5-bf2d-9468488baad2 None None] MFA Rules not processed for user `b3098a7103d94b29bc4bce30cfd05bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:01:54.892592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-5b97f7ce-fb40-49c5-bf2d-9468488baad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:01:55.009583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1188/2374] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:01:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:01:59.953461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:02:03.512579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc3798-c7d8-4f24-88eb-4da78bdb33d3 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:03.513313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc3798-c7d8-4f24-88eb-4da78bdb33d3 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:03.513313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc3798-c7d8-4f24-88eb-4da78bdb33d3 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:03.514913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1187/2375] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 16:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 16:02:03.521266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca8ba2-711d-48a2-9f88-ed01258498f0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:03.521739 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca8ba2-711d-48a2-9f88-ed01258498f0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:03.521827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca8ba2-711d-48a2-9f88-ed01258498f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:03.544942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4eca8ba2-711d-48a2-9f88-ed01258498f0 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:03.546158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4eca8ba2-711d-48a2-9f88-ed01258498f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:03.626285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1189/2376] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 16:02:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:02:03.642722 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc3798-c7d8-4f24-88eb-4da78bdb33d3 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:03.646504 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e7cc3798-c7d8-4f24-88eb-4da78bdb33d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:03.729435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc3798-c7d8-4f24-88eb-4da78bdb33d3 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:03.733018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a22e9a-6b29-47bb-ae2d-7d02b1c36f4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022516f130ec4ad6a9b7c46b417a5690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:03.733619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a22e9a-6b29-47bb-ae2d-7d02b1c36f4d service glance] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:03.734078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a22e9a-6b29-47bb-ae2d-7d02b1c36f4d service glance] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:03.734346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a22e9a-6b29-47bb-ae2d-7d02b1c36f4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:03.744731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a22e9a-6b29-47bb-ae2d-7d02b1c36f4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:03.746736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a22e9a-6b29-47bb-ae2d-7d02b1c36f4d service glance] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:03.757337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1188/2377] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:02:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:02:03.828911 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4466e-80c9-48b8-aa50-6fdc62d32a61 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:03.829507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4466e-80c9-48b8-aa50-6fdc62d32a61 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:03.829878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4466e-80c9-48b8-aa50-6fdc62d32a61 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:03.831489 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1190/2378] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 16:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 16:02:03.836624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f899664-ad98-4c4f-86a7-1a757152d685 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:03.836929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f899664-ad98-4c4f-86a7-1a757152d685 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:03.837198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f899664-ad98-4c4f-86a7-1a757152d685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:03.857620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-8f899664-ad98-4c4f-86a7-1a757152d685 None None] MFA Rules not processed for user `2d007a3d205345aebcc36a36d039da2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:03.858856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8f899664-ad98-4c4f-86a7-1a757152d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:03.913883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1189/2379] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 16:02:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:02:03.926778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4466e-80c9-48b8-aa50-6fdc62d32a61 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:03.928481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-96e4466e-80c9-48b8-aa50-6fdc62d32a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:03.943356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4466e-80c9-48b8-aa50-6fdc62d32a61 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:03.947002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5795b-6d33-40e9-899e-4763b0260db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:03.947848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5795b-6d33-40e9-899e-4763b0260db4 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:03.948141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5795b-6d33-40e9-899e-4763b0260db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:03.948241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5795b-6d33-40e9-899e-4763b0260db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:03.956099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5795b-6d33-40e9-899e-4763b0260db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:03.957543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5795b-6d33-40e9-899e-4763b0260db4 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:03.966338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1191/2380] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:02:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:02:04.080222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d44925-3c13-466b-81ac-78abda60d853 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:04.080508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d44925-3c13-466b-81ac-78abda60d853 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:04.080813 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d44925-3c13-466b-81ac-78abda60d853 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:04.082443 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1190/2381] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 16:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 16:02:04.091208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4238d1a-e954-40b0-bad4-1d9d250568a3 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:04.091496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4238d1a-e954-40b0-bad4-1d9d250568a3 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:04.091771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4238d1a-e954-40b0-bad4-1d9d250568a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:04.112457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e4238d1a-e954-40b0-bad4-1d9d250568a3 None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:04.113720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e4238d1a-e954-40b0-bad4-1d9d250568a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:04.128483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1192/2382] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 16:02:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:02:04.137029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d44925-3c13-466b-81ac-78abda60d853 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:04.149002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d44925-3c13-466b-81ac-78abda60d853 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:04.154776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0630b-8757-479d-b95a-cb6d8ae6e99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:04.155393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0630b-8757-479d-b95a-cb6d8ae6e99a service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:04.155667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0630b-8757-479d-b95a-cb6d8ae6e99a service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:04.155936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0630b-8757-479d-b95a-cb6d8ae6e99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:04.166837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0630b-8757-479d-b95a-cb6d8ae6e99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:04.168747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0630b-8757-479d-b95a-cb6d8ae6e99a service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:04.180450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1191/2383] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:02:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:02:06.266801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daf46-97f4-4359-a5c7-0fc56b6e6319 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:06.267378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daf46-97f4-4359-a5c7-0fc56b6e6319 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:06.267378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daf46-97f4-4359-a5c7-0fc56b6e6319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:06.281825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:02:06.287798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-667daf46-97f4-4359-a5c7-0fc56b6e6319 None None] MFA Rules not processed for user `7377a1bca391446eb36b2a3233a7d471`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:06.289156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-667daf46-97f4-4359-a5c7-0fc56b6e6319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:06.366759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1193/2384] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:02:06.406282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869b7a-9733-4039-b074-d5d0535457b7 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:06.406879 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869b7a-9733-4039-b074-d5d0535457b7 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:06.406976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869b7a-9733-4039-b074-d5d0535457b7 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:06.408621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1192/2385] 10.222.0.33 () {54 vars in 961 bytes} [Mon Oct 3 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 16:02:06.413924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:06.414303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:06.414635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:06.443871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:06.447680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:06.541292 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1194/2386] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Oct 3 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:02:06.549495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26869b7a-9733-4039-b074-d5d0535457b7 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:06.569153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26869b7a-9733-4039-b074-d5d0535457b7 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:06.574379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd8134-355f-45dd-984d-545dfd91425d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:06.575087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd8134-355f-45dd-984d-545dfd91425d service placement] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:06.575427 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd8134-355f-45dd-984d-545dfd91425d service placement] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:06.575803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd8134-355f-45dd-984d-545dfd91425d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:06.577657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8dbd8134-355f-45dd-984d-545dfd91425d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:06.591116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd8134-355f-45dd-984d-545dfd91425d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:06.593023 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd8134-355f-45dd-984d-545dfd91425d service placement] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:06.604234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1193/2387] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:02:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:02:07.845322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:07.849168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:07.921945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e97ddf-ad31-4fd4-a605-554229c6ac8c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:07.928339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:07.929082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:07.929463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:07.929830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] PATH_INFO: `/v3/users/87f51f11bb92407fa0cac3b5702d84bc` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:07.931086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f51f11bb92407fa0cac3b5702d84bc)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:07.937169 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:08.005158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f51f11bb92407fa0cac3b5702d84bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:08.008033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:02:08.017201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f51f11bb92407fa0cac3b5702d84bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:08.033958 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f51f11bb92407fa0cac3b5702d84bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:08.042778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f51f11bb92407fa0cac3b5702d84bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:08.043632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f51f11bb92407fa0cac3b5702d84bc'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:08.052777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1195/2388] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:02:07 2022] DELETE /identity/v3/users/87f51f11bb92407fa0cac3b5702d84bc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:02:08.056910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:08.058406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:08.058704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:08.096149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:08.097399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:08.183736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1194/2389] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:02:08.194449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:08.196019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:08.267077 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfae41-8ae2-4663-ad32-80fe0410b585 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:08.271154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c319f-0902-48c9-a875-66fe41465c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:08.271824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c319f-0902-48c9-a875-66fe41465c9d service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:08.272257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c319f-0902-48c9-a875-66fe41465c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:08.272378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c319f-0902-48c9-a875-66fe41465c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:08.282568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c319f-0902-48c9-a875-66fe41465c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:08.284017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c319f-0902-48c9-a875-66fe41465c9d service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:08.297329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1196/2390] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:02:08.908052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:08.909728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:08.934196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4a02f-600b-4576-b387-17fcf4dba691 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:08.939733 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:08.940298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:08.940819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:08.941075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] PATH_INFO: `/v3/projects/df40819799cf4e0cb9f8bb69ee6e6813` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:08.942471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=df40819799cf4e0cb9f8bb69ee6e6813)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:08.944769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:08.983253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:02:09.038698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df40819799cf4e0cb9f8bb69ee6e6813', 'request_id': 'req-3b331cd8-b9ec-4087-98fc-91fae7f69fca'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:09.057549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df40819799cf4e0cb9f8bb69ee6e6813', 'request_id': 'req-3b331cd8-b9ec-4087-98fc-91fae7f69fca'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:09.071824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df40819799cf4e0cb9f8bb69ee6e6813', 'request_id': 'req-3b331cd8-b9ec-4087-98fc-91fae7f69fca'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:09.109678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] The token cache is being invalidate because project df40819799cf4e0cb9f8bb69ee6e6813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:02:09.110302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:09.111228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:02:09.113664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1195/2391] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:02:08 2022] DELETE /identity/v3/projects/df40819799cf4e0cb9f8bb69ee6e6813 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:02:10.029724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeb88b-c8a9-45d6-8103-fe17e8bba345 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:10.030208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeb88b-c8a9-45d6-8103-fe17e8bba345 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:10.030323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aeb88b-c8a9-45d6-8103-fe17e8bba345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:10.066737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-50aeb88b-c8a9-45d6-8103-fe17e8bba345 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:10.068022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-50aeb88b-c8a9-45d6-8103-fe17e8bba345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:10.152259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1197/2392] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:02:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:02:10.159934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:10.161571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:10.258736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b331cd8-b9ec-4087-98fc-91fae7f69fca admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:10.264928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:10.265541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:10.265975 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:10.266414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:10.279509 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:10.281238 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:10.290140 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1196/2393] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:02:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:02:20.136713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=3,keystone:SELECT=63 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:02:20.286506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:DELETE=7,keystone:SELECT=47,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:02:36.311064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:36.311621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:36.311621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:36.335553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:02:36.336875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:36.345256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1198/2394] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 16:02:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:02:36.369255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:36.380103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e2ce-d55e-479d-9d41-0d5502b23f9f service swift] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:36.385328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bb757-3318-47c5-9d5e-6fba7b906fb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:36.385920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb757-3318-47c5-9d5e-6fba7b906fb6 service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:36.386267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb757-3318-47c5-9d5e-6fba7b906fb6 service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:36.386617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bb757-3318-47c5-9d5e-6fba7b906fb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:36.395846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bb757-3318-47c5-9d5e-6fba7b906fb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:36.397794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42bb757-3318-47c5-9d5e-6fba7b906fb6 service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:36.406622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1197/2395] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:02:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:02:37.884647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:02:37.886186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:37.943259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd512a8f-fa61-43dc-8b49-85c1d99ad4d9 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:02:37.946839 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbb76e-50d9-48b4-9ff8-7d90023ad24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:02:37.947433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbb76e-50d9-48b4-9ff8-7d90023ad24b service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:02:37.947709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbb76e-50d9-48b4-9ff8-7d90023ad24b service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:02:37.947977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbb76e-50d9-48b4-9ff8-7d90023ad24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:02:37.949718 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-40bbb76e-50d9-48b4-9ff8-7d90023ad24b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:02:38.011674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbb76e-50d9-48b4-9ff8-7d90023ad24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:02:38.013151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbb76e-50d9-48b4-9ff8-7d90023ad24b service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:02:38.023296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1199/2396] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:02:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:02:46.402961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=6 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:02:48.018780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=53 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:03:20.336285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d966765-d829-4d6d-929c-1d300ce2c0e8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:03:20.336794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d966765-d829-4d6d-929c-1d300ce2c0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:03:20.336897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d966765-d829-4d6d-929c-1d300ce2c0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:03:20.356022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5d966765-d829-4d6d-929c-1d300ce2c0e8 None None] MFA Rules not processed for user `5d59791fcdee41149fce9b8a950963fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:03:20.357220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5d966765-d829-4d6d-929c-1d300ce2c0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:03:20.410692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1198/2397] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:03:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:03:30.404338 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=25 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:03:42.217198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad05e6-3e53-436a-aec5-51103b05f6bd None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:03:42.218143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad05e6-3e53-436a-aec5-51103b05f6bd None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:03:42.218584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad05e6-3e53-436a-aec5-51103b05f6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:03:42.239198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-82ad05e6-3e53-436a-aec5-51103b05f6bd None None] MFA Rules not processed for user `5e113c9ec968460ba070cfc1a69a25a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:03:42.241149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-82ad05e6-3e53-436a-aec5-51103b05f6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:03:42.297337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1200/2398] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:03:48.256095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7187c6d-c0c3-49b6-b3a6-127728b418d0 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:03:48.257300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7187c6d-c0c3-49b6-b3a6-127728b418d0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:03:48.257577 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7187c6d-c0c3-49b6-b3a6-127728b418d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:03:48.281472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-b7187c6d-c0c3-49b6-b3a6-127728b418d0 None None] MFA Rules not processed for user `b3098a7103d94b29bc4bce30cfd05bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:03:48.283662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b7187c6d-c0c3-49b6-b3a6-127728b418d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:03:48.360558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1199/2399] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:03:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:03:50.901855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdfe0d-de83-4b45-88bc-cd9ceaa80bfe None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:03:50.902387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdfe0d-de83-4b45-88bc-cd9ceaa80bfe None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:03:50.902490 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdfe0d-de83-4b45-88bc-cd9ceaa80bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:03:50.922797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a4bdfe0d-de83-4b45-88bc-cd9ceaa80bfe None None] MFA Rules not processed for user `73a1aa0996db4c8ea851b2ba45026b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:03:50.923978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a4bdfe0d-de83-4b45-88bc-cd9ceaa80bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:03:50.978951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1201/2400] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:03:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:03:58.352162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=25 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:03:59.856211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7187c6d-c0c3-49b6-b3a6-127728b418d0 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:03:59.868759 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7187c6d-c0c3-49b6-b3a6-127728b418d0 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:03:59.875799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479bcc6-e9ab-419a-b203-e84b5b095f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:03:59.876430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479bcc6-e9ab-419a-b203-e84b5b095f0e service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:03:59.876700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479bcc6-e9ab-419a-b203-e84b5b095f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:03:59.877350 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479bcc6-e9ab-419a-b203-e84b5b095f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:03:59.879141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3479bcc6-e9ab-419a-b203-e84b5b095f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:03:59.962622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479bcc6-e9ab-419a-b203-e84b5b095f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:03:59.964346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479bcc6-e9ab-419a-b203-e84b5b095f0e service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:03:59.976585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1200/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:03:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:04:00.971860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=50 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:01.696376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdfe0d-de83-4b45-88bc-cd9ceaa80bfe None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:01.706022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdfe0d-de83-4b45-88bc-cd9ceaa80bfe None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:01.709604 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfccd9-0dc3-45c6-97c1-5e6fe8f1f39c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:01.710195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfccd9-0dc3-45c6-97c1-5e6fe8f1f39c service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:01.710485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfccd9-0dc3-45c6-97c1-5e6fe8f1f39c service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:01.710751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfccd9-0dc3-45c6-97c1-5e6fe8f1f39c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:01.712476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-27bfccd9-0dc3-45c6-97c1-5e6fe8f1f39c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:01.726633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfccd9-0dc3-45c6-97c1-5e6fe8f1f39c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:01.728044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfccd9-0dc3-45c6-97c1-5e6fe8f1f39c service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:01.738635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1202/2402] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:05.051412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeced5f-52f6-4488-adb5-71db67f8cb38 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:05.052666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeced5f-52f6-4488-adb5-71db67f8cb38 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:05.052666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeced5f-52f6-4488-adb5-71db67f8cb38 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:05.055360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1201/2403] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 16:04:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 16:04:05.066909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e607aa-380e-488e-9dea-733571b0597d None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:05.068175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e607aa-380e-488e-9dea-733571b0597d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:05.068175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e607aa-380e-488e-9dea-733571b0597d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:05.102196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-92e607aa-380e-488e-9dea-733571b0597d None None] MFA Rules not processed for user `2d007a3d205345aebcc36a36d039da2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:04:05.103996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-92e607aa-380e-488e-9dea-733571b0597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:05.115445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1203/2404] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 16:04:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:04:05.134228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeced5f-52f6-4488-adb5-71db67f8cb38 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:05.136569 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-afeced5f-52f6-4488-adb5-71db67f8cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:05.160803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeced5f-52f6-4488-adb5-71db67f8cb38 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:05.166030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea4250-07d0-4f6a-8fd0-bd04a68f61a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:05.166802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea4250-07d0-4f6a-8fd0-bd04a68f61a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:05.167078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea4250-07d0-4f6a-8fd0-bd04a68f61a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:05.167523 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea4250-07d0-4f6a-8fd0-bd04a68f61a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:05.181956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea4250-07d0-4f6a-8fd0-bd04a68f61a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:05.183973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea4250-07d0-4f6a-8fd0-bd04a68f61a4 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:05.194915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1202/2405] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:05.396755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bf4ff-0b4e-4584-8b6e-48b3c07e5bc0 None None] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:05.397235 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bf4ff-0b4e-4584-8b6e-48b3c07e5bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:05.397666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bf4ff-0b4e-4584-8b6e-48b3c07e5bc0 None None] PATH_INFO: `/` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:05.399409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1204/2406] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 16:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 16:04:05.405413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56665acb-8897-41a8-96d6-d6da7a0f1adc None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:05.405896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56665acb-8897-41a8-96d6-d6da7a0f1adc None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:05.406166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56665acb-8897-41a8-96d6-d6da7a0f1adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:05.440714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-56665acb-8897-41a8-96d6-d6da7a0f1adc None None] MFA Rules not processed for user `ed959e4bdaf44132b05ccaec68c41172`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:04:05.443058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-56665acb-8897-41a8-96d6-d6da7a0f1adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:05.459188 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1203/2407] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 16:04:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:04:05.479624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bf4ff-0b4e-4584-8b6e-48b3c07e5bc0 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:05.499985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bf4ff-0b4e-4584-8b6e-48b3c07e5bc0 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:05.505468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60910b-be94-4454-9ef4-948a0445a624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:05.507171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60910b-be94-4454-9ef4-948a0445a624 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:05.508018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60910b-be94-4454-9ef4-948a0445a624 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:05.508852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60910b-be94-4454-9ef4-948a0445a624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:05.538091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60910b-be94-4454-9ef4-948a0445a624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:05.540529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60910b-be94-4454-9ef4-948a0445a624 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:05.554921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1205/2408] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:15.434069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=43 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:15.550093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=15 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:28.598614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aba3-4e06-48f1-99f6-b7f5f4e41422 None None] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:28.599136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aba3-4e06-48f1-99f6-b7f5f4e41422 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:28.599900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622aba3-4e06-48f1-99f6-b7f5f4e41422 None None] PATH_INFO: `/` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:28.601341 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1204/2409] 10.222.0.33 () {54 vars in 921 bytes} [Mon Oct 3 16:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 16:04:28.625864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:28.626357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:28.626455 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:28.647168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] MFA Rules not processed for user `92a197dc8b2749e08aa0d0612996cb1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:04:28.648397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:28.670323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622aba3-4e06-48f1-99f6-b7f5f4e41422 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:28.672235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1622aba3-4e06-48f1-99f6-b7f5f4e41422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:28.716702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1206/2410] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Oct 3 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:04:28.728593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:28.729949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:28.740261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622aba3-4e06-48f1-99f6-b7f5f4e41422 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:28.742324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6287085-42f3-4c81-a082-a1e2152dc306 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:28.745281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:28.746022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:28.746308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:28.746581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] PATH_INFO: `/v3/users/5d59791fcdee41149fce9b8a950963fc` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:28.747488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:28.747745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d59791fcdee41149fce9b8a950963fc)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:28.748072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:28.748343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:28.748607 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:28.750417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:28.757488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:28.759046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:28.768537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1207/2411] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:04:28.804137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d59791fcdee41149fce9b8a950963fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:28.804829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d59791fcdee41149fce9b8a950963fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:28.812685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d59791fcdee41149fce9b8a950963fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:28.827931 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d59791fcdee41149fce9b8a950963fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:28.836395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d59791fcdee41149fce9b8a950963fc'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:28.843276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1205/2412] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:04:28 2022] DELETE /identity/v3/users/5d59791fcdee41149fce9b8a950963fc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:04:29.013457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:29.024758 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa1d38-a80e-432e-97d0-60e10f65f138 service placement] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:29.095695 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:29.096330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:29.096610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:29.096915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] PATH_INFO: `/v3/projects/f8c41c14b0b3411d9b7e39e3983f2ccd` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:29.098126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c41c14b0b3411d9b7e39e3983f2ccd)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:29.100526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:29.147787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c41c14b0b3411d9b7e39e3983f2ccd', 'request_id': 'req-cdfc80ad-d397-4807-a079-7e439b038685'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:29.154202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c41c14b0b3411d9b7e39e3983f2ccd', 'request_id': 'req-cdfc80ad-d397-4807-a079-7e439b038685'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:29.159715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c41c14b0b3411d9b7e39e3983f2ccd', 'request_id': 'req-cdfc80ad-d397-4807-a079-7e439b038685'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:29.170526 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] The token cache is being invalidate because project f8c41c14b0b3411d9b7e39e3983f2ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:04:29.170812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:29.171445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:29.172968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1208/2413] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:04:29 2022] DELETE /identity/v3/projects/f8c41c14b0b3411d9b7e39e3983f2ccd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:04:31.330176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:31.331641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:31.387856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41827f23-81f8-4b0f-ad81-6b05ec005f68 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:31.418673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:31.436941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:31.437536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:31.437811 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:31.438078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:31.439815 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:31.497484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:31.498915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:31.508297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1206/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:39.167609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:41.503631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=32 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:44.835464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:44.836929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:44.919315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc80ad-d397-4807-a079-7e439b038685 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:44.924794 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:44.925424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:44.925692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:44.925981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:44.927863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:44.940702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:44.942448 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:44.953650 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1209/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:04:49.095475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:49.097015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:49.112971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec263b-7f48-4b0d-aef9-e24a29de7315 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:49.119068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:49.119663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:49.119934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:49.120202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:49.121919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:49.182158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:49.183748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:49.193705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1207/2416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:51.352935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:51.354972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:51.415584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8f750-743b-4fc1-bbab-2199845896ce service nova] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:51.419395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:51.419993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:51.420267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:51.420536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:51.422633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:51.487851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:51.490083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:51.500019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1210/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:54.900508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:54.901980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:54.982791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:54.984373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:55.003734 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4392bc3-3711-4b79-9800-b674b63046ab service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.007674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d1a30-946d-45cd-a2ef-5160a602d503 service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.011927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.012686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.013002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.013276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] PATH_INFO: `/v3/users/b3098a7103d94b29bc4bce30cfd05bc6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.013528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.014148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.014422 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3098a7103d94b29bc4bce30cfd05bc6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.014550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.014719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.016391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.016525 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:55.031412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.033456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.052843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1208/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:55.063176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:55.080840 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3098a7103d94b29bc4bce30cfd05bc6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.090651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3098a7103d94b29bc4bce30cfd05bc6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.106171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3098a7103d94b29bc4bce30cfd05bc6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.115301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3098a7103d94b29bc4bce30cfd05bc6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.116015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-065abe96-56c8-440a-b7fe-432561570b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3098a7103d94b29bc4bce30cfd05bc6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.122864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1211/2419] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:04:54 2022] DELETE /identity/v3/users/b3098a7103d94b29bc4bce30cfd05bc6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:04:55.341883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:55.355004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25018c52-1dcb-4206-8ab4-44dec9010213 service placement] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.412573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.413232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.413512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.413806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] PATH_INFO: `/v3/projects/f2ed0fa1c2fe47a485c26522b2e81b8b` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.414932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ed0fa1c2fe47a485c26522b2e81b8b)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.416883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.449627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:55.463437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ed0fa1c2fe47a485c26522b2e81b8b', 'request_id': 'req-f62be621-013e-437e-adc0-64362a1d9d38'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.470563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ed0fa1c2fe47a485c26522b2e81b8b', 'request_id': 'req-f62be621-013e-437e-adc0-64362a1d9d38'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.476921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ed0fa1c2fe47a485c26522b2e81b8b', 'request_id': 'req-f62be621-013e-437e-adc0-64362a1d9d38'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.487694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] The token cache is being invalidate because project f2ed0fa1c2fe47a485c26522b2e81b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:04:55.487980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.488595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f62be621-013e-437e-adc0-64362a1d9d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:04:55.490109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1209/2420] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:04:55 2022] DELETE /identity/v3/projects/f2ed0fa1c2fe47a485c26522b2e81b8b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:04:55.495182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325209c5-6af3-4b76-855d-c962215bf63b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.495470 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325209c5-6af3-4b76-855d-c962215bf63b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.495749 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325209c5-6af3-4b76-855d-c962215bf63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.515058 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-325209c5-6af3-4b76-855d-c962215bf63b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:04:55.516273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-325209c5-6af3-4b76-855d-c962215bf63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:55.576623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1212/2421] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:04:55.585310 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516022f7-1d0b-4be9-8a93-627cf713c05e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.585606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516022f7-1d0b-4be9-8a93-627cf713c05e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.585864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516022f7-1d0b-4be9-8a93-627cf713c05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.606438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-516022f7-1d0b-4be9-8a93-627cf713c05e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:04:55.607640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-516022f7-1d0b-4be9-8a93-627cf713c05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:55.615493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1210/2422] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:04:55.619871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325209c5-6af3-4b76-855d-c962215bf63b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:55.630027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325209c5-6af3-4b76-855d-c962215bf63b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.634679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.635273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.635548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.635814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.636488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:04:55.636822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.638426 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.652694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1213/2423] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:04:55.657158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516022f7-1d0b-4be9-8a93-627cf713c05e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:55.666306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516022f7-1d0b-4be9-8a93-627cf713c05e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.670996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.671591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.671864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.672130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.672790 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.674694 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.701793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1211/2424] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:04:55 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:04:55.706043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:55.716742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c213b-ea06-43a1-9151-9e78a08ccd75 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.774872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.775476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.775752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.776019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.776652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.778587 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.800445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1214/2425] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:04:55 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:04:55.805090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:55.815231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39cfb4-eaeb-434d-9729-a80f199945f0 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.820568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.821196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.821473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.821751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.822396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.824007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.835968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1212/2426] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:04:55.840192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:55.850875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e009e2b-0855-41a5-8eb7-6b69b30afb1e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.855507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162e860-495c-4e49-9ad4-06340218ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.856100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162e860-495c-4e49-9ad4-06340218ef67 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.856384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162e860-495c-4e49-9ad4-06340218ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.856656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162e860-495c-4e49-9ad4-06340218ef67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.857331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162e860-495c-4e49-9ad4-06340218ef67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.858953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162e860-495c-4e49-9ad4-06340218ef67 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.869565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1215/2427] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:04:55.873779 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:55.884324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2c8f9-e09d-4429-a4ed-0cd4620c06c1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:55.888869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:55.889501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.889782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.890041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] PATH_INFO: `/v3/projects/62932ad37df941b1840b183a858c7876/users/4cc1db8ef8c44763a0e5f62a3404a834/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.904859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62932ad37df941b1840b183a858c7876, user_id=4cc1db8ef8c44763a0e5f62a3404a834, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:55.907488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:55.919700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1213/2428] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:04:55 2022] PUT /identity/v3/projects/62932ad37df941b1840b183a858c7876/users/4cc1db8ef8c44763a0e5f62a3404a834/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:04:55.924790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93200-9262-4415-895d-552d66d4b433 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:55.925845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93200-9262-4415-895d-552d66d4b433 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:55.925845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d93200-9262-4415-895d-552d66d4b433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:55.945182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-64d93200-9262-4415-895d-552d66d4b433 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:04:55.946379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-64d93200-9262-4415-895d-552d66d4b433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:55.954459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:04:56.006871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1216/2429] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:04:56.016351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:56.018144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:56.075070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e527e8-0293-4ed4-842d-ef74b608ea65 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:56.078696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e93b5a-3534-4c7e-a00a-ee35ebb11f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:56.079289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93b5a-3534-4c7e-a00a-ee35ebb11f9c service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:56.079567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93b5a-3534-4c7e-a00a-ee35ebb11f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:56.079833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93b5a-3534-4c7e-a00a-ee35ebb11f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:56.089099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e93b5a-3534-4c7e-a00a-ee35ebb11f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:56.090483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e93b5a-3534-4c7e-a00a-ee35ebb11f9c service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:56.099654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1214/2430] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:04:57.140927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d93200-9262-4415-895d-552d66d4b433 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:04:57.142454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-64d93200-9262-4415-895d-552d66d4b433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:57.238157 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d93200-9262-4415-895d-552d66d4b433 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:04:57.244965 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:04:57.245664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:04:57.246017 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:04:57.246362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:04:57.248287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:04:57.326154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:04:57.327756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:04:57.338439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1217/2431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:04:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:01.434402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c66f2-8d0b-4141-abf7-1cf6d8a20016 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:01.434972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c66f2-8d0b-4141-abf7-1cf6d8a20016 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:01.435307 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c66f2-8d0b-4141-abf7-1cf6d8a20016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:01.468517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-fb4c66f2-8d0b-4141-abf7-1cf6d8a20016 None None] MFA Rules not processed for user `4cc1db8ef8c44763a0e5f62a3404a834`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:01.470197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fb4c66f2-8d0b-4141-abf7-1cf6d8a20016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:01.530250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:01.563738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1215/2432] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:01.579811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:01.585580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:01.650264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fc4e8-f48c-4073-8144-8257d489ec13 service nova] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:01.672918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33755ab3-81a0-4848-9ce5-bb1b1a2b7dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:01.673574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33755ab3-81a0-4848-9ce5-bb1b1a2b7dff service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:01.673852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33755ab3-81a0-4848-9ce5-bb1b1a2b7dff service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:01.674123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33755ab3-81a0-4848-9ce5-bb1b1a2b7dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:01.703814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33755ab3-81a0-4848-9ce5-bb1b1a2b7dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:01.706742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33755ab3-81a0-4848-9ce5-bb1b1a2b7dff service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:01.722549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1218/2433] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:02.785469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:02.785900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:02.786161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:02.807716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] MFA Rules not processed for user `7377a1bca391446eb36b2a3233a7d471`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:02.811958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:02.887425 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1216/2434] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:03.445268 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75855384-f5c1-4041-af8e-1d93e9587d2f None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:03.445776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75855384-f5c1-4041-af8e-1d93e9587d2f None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:03.445776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75855384-f5c1-4041-af8e-1d93e9587d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:03.449588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:03.451288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:03.466930 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-75855384-f5c1-4041-af8e-1d93e9587d2f None None] MFA Rules not processed for user `4cc1db8ef8c44763a0e5f62a3404a834`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:03.467754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dc6d4-2098-4c8e-86a7-4987519b3aaa None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:03.468574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-75855384-f5c1-4041-af8e-1d93e9587d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:03.472123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:03.473491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:03.473871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:03.474144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:03.476493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:03.479677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1219/2435] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:03.489183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:03.490785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:03.499800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1217/2436] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:05:03.925464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75855384-f5c1-4041-af8e-1d93e9587d2f None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:03.939264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75855384-f5c1-4041-af8e-1d93e9587d2f None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:03.944474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:03.945185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:03.945472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:03.945822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:03.947539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:03.959193 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:03.960743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:03.978590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1220/2437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:04.149700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:04.151111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:04.246543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14777eae-2769-439a-b049-1c2217a60429 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:04.259700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:04.260323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:04.260812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:04.261255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:04.265847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:04.282883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:04.286189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:04.318959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1218/2438] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:07.323106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:07.332819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b3288-92db-4496-8c47-86712ed73874 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:07.336549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e442393-7c1f-47d8-b12d-9194966e3b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:07.337185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e442393-7c1f-47d8-b12d-9194966e3b76 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:07.337462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e442393-7c1f-47d8-b12d-9194966e3b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:07.338224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e442393-7c1f-47d8-b12d-9194966e3b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:07.340030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0e442393-7c1f-47d8-b12d-9194966e3b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:07.352068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e442393-7c1f-47d8-b12d-9194966e3b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:07.354316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e442393-7c1f-47d8-b12d-9194966e3b76 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:07.364408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1221/2439] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:10.075543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:10.091595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356fb73-8f01-4857-b67c-ee85230801a6 service placement] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:10.098750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:10.099854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:10.101067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:10.101562 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:10.105190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:10.119922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:10.122006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:10.135342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1219/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:10.240409 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32f57-5a8f-4bf6-94aa-621b36724d22 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:10.240949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32f57-5a8f-4bf6-94aa-621b36724d22 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:10.241046 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32f57-5a8f-4bf6-94aa-621b36724d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:10.249567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:10.263847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b5b32f57-5a8f-4bf6-94aa-621b36724d22 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:10.265347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b5b32f57-5a8f-4bf6-94aa-621b36724d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:10.347699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1222/2441] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:05:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:10.405819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:10.407239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:10.471712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:10.492756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62de8d-f41a-409e-b758-fee494d49f80 service nova] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:10.498484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2cf7b-7f37-430b-9d4e-c4ce05cc0342 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:10.499085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2cf7b-7f37-430b-9d4e-c4ce05cc0342 service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:10.499359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2cf7b-7f37-430b-9d4e-c4ce05cc0342 service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:10.499625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2cf7b-7f37-430b-9d4e-c4ce05cc0342 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:10.509058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2cf7b-7f37-430b-9d4e-c4ce05cc0342 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:10.511388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e2cf7b-7f37-430b-9d4e-c4ce05cc0342 service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:10.522410 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1220/2442] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:05:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:05:12.279844 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b32f57-5a8f-4bf6-94aa-621b36724d22 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:12.297113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b32f57-5a8f-4bf6-94aa-621b36724d22 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:12.319797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:12.321906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:12.322910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:12.323273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:12.326529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:12.344515 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:12.346702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:12.360105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1223/2443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:20.516922 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=7 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:22.355164 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=29 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:30.638621 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842ec8c-18fd-4ca0-a057-1a4e95d69e78 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:30.639087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842ec8c-18fd-4ca0-a057-1a4e95d69e78 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:30.639189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842ec8c-18fd-4ca0-a057-1a4e95d69e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:30.659281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-0842ec8c-18fd-4ca0-a057-1a4e95d69e78 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:30.660502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0842ec8c-18fd-4ca0-a057-1a4e95d69e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:30.667885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1221/2444] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:05:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:30.679852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:30.691722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eb849-61ea-4b5e-a2cc-32b11fc2ab95 service nova] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:30.696652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:30.697307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:30.697611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:30.697878 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:30.707673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:30.709275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:30.717319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1224/2445] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:05:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:05:32.609031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842ec8c-18fd-4ca0-a057-1a4e95d69e78 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:32.619913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842ec8c-18fd-4ca0-a057-1a4e95d69e78 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:32.623662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:32.624277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:32.624555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:32.624882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:32.626723 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:32.640625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:32.642134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:32.652609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1222/2446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:36.597041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:36.598621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:36.625384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe5edb-fd98-454b-99eb-3edb88494ba7 service swift] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:36.631144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:36.631920 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:36.632204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:36.632527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] PATH_INFO: `/v3/users/73a1aa0996db4c8ea851b2ba45026b35` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:36.633712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a1aa0996db4c8ea851b2ba45026b35)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:36.635835 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:36.692055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a1aa0996db4c8ea851b2ba45026b35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:36.704000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a1aa0996db4c8ea851b2ba45026b35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:36.717846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a1aa0996db4c8ea851b2ba45026b35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:36.725572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a1aa0996db4c8ea851b2ba45026b35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:36.726382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-9e8cb0ac-50c9-4a25-8a72-5e216e4ff483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a1aa0996db4c8ea851b2ba45026b35'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:36.735199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1225/2447] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:05:36 2022] DELETE /identity/v3/users/73a1aa0996db4c8ea851b2ba45026b35 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:36.950146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:36.965993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe34e6-612a-41ee-b25f-4cbf6ff8461a service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.045434 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.046226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.046512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.046785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] PATH_INFO: `/v3/projects/a8db1f811cf84507ad7ea7058d6a0d25` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.047937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8db1f811cf84507ad7ea7058d6a0d25)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.050090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.106941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8db1f811cf84507ad7ea7058d6a0d25', 'request_id': 'req-a0901d24-206d-4d01-bdce-2034432f64c6'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:37.116489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8db1f811cf84507ad7ea7058d6a0d25', 'request_id': 'req-a0901d24-206d-4d01-bdce-2034432f64c6'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:37.123204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8db1f811cf84507ad7ea7058d6a0d25', 'request_id': 'req-a0901d24-206d-4d01-bdce-2034432f64c6'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:37.136567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] The token cache is being invalidate because project a8db1f811cf84507ad7ea7058d6a0d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:05:37.137037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:37.137845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a0901d24-206d-4d01-bdce-2034432f64c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:37.140173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1223/2448] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:05:36 2022] DELETE /identity/v3/projects/a8db1f811cf84507ad7ea7058d6a0d25 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:37.145841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a497b-f6c6-4d84-9b8b-3e460c0e2aae None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.146379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a497b-f6c6-4d84-9b8b-3e460c0e2aae None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.146849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a497b-f6c6-4d84-9b8b-3e460c0e2aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.167809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6f5a497b-f6c6-4d84-9b8b-3e460c0e2aae None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:37.169140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6f5a497b-f6c6-4d84-9b8b-3e460c0e2aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:37.234471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1226/2449] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:37.243287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db03046-df7e-4246-af8c-3bee1952c211 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.243627 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db03046-df7e-4246-af8c-3bee1952c211 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.243881 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db03046-df7e-4246-af8c-3bee1952c211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.276122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6db03046-df7e-4246-af8c-3bee1952c211 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:37.277376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6db03046-df7e-4246-af8c-3bee1952c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:37.289319 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1224/2450] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:37.290531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a497b-f6c6-4d84-9b8b-3e460c0e2aae None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.301912 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a497b-f6c6-4d84-9b8b-3e460c0e2aae None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.309137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.309766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.310037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.310539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.311203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:05:37.311504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.313168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.327541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1227/2451] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:05:37.332056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db03046-df7e-4246-af8c-3bee1952c211 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.341792 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db03046-df7e-4246-af8c-3bee1952c211 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.347423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.348013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.348285 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.348546 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.349893 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.353148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.399092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1225/2452] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:05:37 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:37.402951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.414229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f25c2b-ee6a-4510-bcf5-51b3301ac24f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.479821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:37.503121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.503891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.504227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.504561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.505331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.507677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.536953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1228/2453] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:05:37 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:37.540981 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.552409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66b63-6fd4-45d5-abbe-1b6745a1596b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.558094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.558712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.559146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.559423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.560040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.561667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.573915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1226/2454] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:37.580904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.606346 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257336d-c94b-417a-be85-f929c1235b00 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.612009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff107ff-f054-4b4d-930d-f26e94e78d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.612633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff107ff-f054-4b4d-930d-f26e94e78d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.612943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff107ff-f054-4b4d-930d-f26e94e78d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.613217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff107ff-f054-4b4d-930d-f26e94e78d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.613870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff107ff-f054-4b4d-930d-f26e94e78d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.615481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff107ff-f054-4b4d-930d-f26e94e78d5d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.627837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1229/2455] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:37.631983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.643796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e04b-ab06-4bb1-8980-e576776c4b08 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.649748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.650617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.650871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.651146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] PATH_INFO: `/v3/projects/d2a8900a1b484daa9d3c42d3c619efc7/users/bf72061e52d84858923219f66b980c02/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.667605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a8900a1b484daa9d3c42d3c619efc7, user_id=bf72061e52d84858923219f66b980c02, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.670346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.685202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1227/2456] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:37 2022] PUT /identity/v3/projects/d2a8900a1b484daa9d3c42d3c619efc7/users/bf72061e52d84858923219f66b980c02/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:05:37.697001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9612-5e94-48d1-9460-6a67e32438e7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.697386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9612-5e94-48d1-9460-6a67e32438e7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.697757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9612-5e94-48d1-9460-6a67e32438e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.728383 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-10ac9612-5e94-48d1-9460-6a67e32438e7 None None] MFA Rules not processed for user `bf72061e52d84858923219f66b980c02`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:37.730297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-10ac9612-5e94-48d1-9460-6a67e32438e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:37.818575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1230/2457] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:37.822820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.832134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:37.899445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837e3c2-f279-4c1e-9115-4a6218cfc2ce admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:37.904417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:37.905055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:37.905326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:37.905594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:37.906456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:37.909066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:37.946037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1228/2458] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:05:37 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:37.947429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac9612-5e94-48d1-9460-6a67e32438e7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:37.960291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac9612-5e94-48d1-9460-6a67e32438e7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:38.028554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:38.029186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:38.029466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:38.029909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:38.030648 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:38.032926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:38.060565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1231/2459] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:05:37 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:05:38.063155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:38.073867 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08781dcf-7261-44ed-93f3-600333776d53 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:38.078837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:38.079464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:38.079730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:38.080041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:38.081604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:38.083402 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:38.098699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1229/2460] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:38.103334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:38.113923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d9ece-8ef7-4bee-bff4-ee9868df0f94 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:38.119781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:38.120416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:38.120796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:38.121133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] PATH_INFO: `/v3/projects/72c6a91ca74c484badfa941b135c848c/users/9c6a97513a064f279535c3fb7b491531/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:38.137197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c6a91ca74c484badfa941b135c848c, user_id=9c6a97513a064f279535c3fb7b491531, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:38.140069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:38.157000 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1232/2461] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:38 2022] PUT /identity/v3/projects/72c6a91ca74c484badfa941b135c848c/users/9c6a97513a064f279535c3fb7b491531/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:38.161737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:38.174093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1762e42-c6a6-4756-8967-66b7f69f7fc6 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:38.251736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef0bf8-3ed0-41a4-a31f-1497f61f604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:38.252481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef0bf8-3ed0-41a4-a31f-1497f61f604a admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:38.252952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef0bf8-3ed0-41a4-a31f-1497f61f604a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:38.253198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef0bf8-3ed0-41a4-a31f-1497f61f604a admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:38.253967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef0bf8-3ed0-41a4-a31f-1497f61f604a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:38.255611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef0bf8-3ed0-41a4-a31f-1497f61f604a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:38.266559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1230/2462] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:05:38.271395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:38.282813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ccc20-0846-4d58-908f-750f06944353 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:38.294761 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:38.295439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:38.295712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:38.296004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] PATH_INFO: `/v3/projects/72c6a91ca74c484badfa941b135c848c/users/9c6a97513a064f279535c3fb7b491531/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:38.297982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c6a91ca74c484badfa941b135c848c, user_id=9c6a97513a064f279535c3fb7b491531, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:38.303431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:38.322135 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1233/2463] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:38 2022] PUT /identity/v3/projects/72c6a91ca74c484badfa941b135c848c/users/9c6a97513a064f279535c3fb7b491531/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:38.333041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e028d3-90d9-4f5a-aef1-20c6cc8ac5dd None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:38.333331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e028d3-90d9-4f5a-aef1-20c6cc8ac5dd None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:38.333604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e028d3-90d9-4f5a-aef1-20c6cc8ac5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:38.367143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-52e028d3-90d9-4f5a-aef1-20c6cc8ac5dd None None] MFA Rules not processed for user `9c6a97513a064f279535c3fb7b491531`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:38.368620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-52e028d3-90d9-4f5a-aef1-20c6cc8ac5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:38.446320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1231/2464] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:38.458231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:38.460031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:38.472182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:38.528374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3779fb5-ca8c-4f37-a820-c31b62d1b162 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:38.532063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2d6cb-0f37-47d7-ad58-3a20a599ba6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:38.532711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2d6cb-0f37-47d7-ad58-3a20a599ba6e service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:38.533293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2d6cb-0f37-47d7-ad58-3a20a599ba6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:38.533568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2d6cb-0f37-47d7-ad58-3a20a599ba6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:38.551173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2d6cb-0f37-47d7-ad58-3a20a599ba6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:38.552811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2d6cb-0f37-47d7-ad58-3a20a599ba6e service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:38.564121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1234/2465] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:39.124271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e028d3-90d9-4f5a-aef1-20c6cc8ac5dd None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:39.129510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:39.202968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e028d3-90d9-4f5a-aef1-20c6cc8ac5dd None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:39.209921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:39.210561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:39.210835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:39.211106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] PATH_INFO: `/v3/users/bf72061e52d84858923219f66b980c02` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:39.212498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf72061e52d84858923219f66b980c02)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:39.214505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:39.264144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf72061e52d84858923219f66b980c02'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.264748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf72061e52d84858923219f66b980c02'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.272942 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf72061e52d84858923219f66b980c02'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.290451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf72061e52d84858923219f66b980c02'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.297730 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf72061e52d84858923219f66b980c02'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.308990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1232/2466] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:05:39 2022] DELETE /identity/v3/users/bf72061e52d84858923219f66b980c02 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:39.313497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:39.314123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:39.314123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:39.333573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:39.334795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:39.399731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1235/2467] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:39.410370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:39.411961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:39.472748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd80fa-f5d9-46ad-837e-dec2d2511d5c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:39.476494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:39.477150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:39.477429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:39.477781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:39.487859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:39.489353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:39.498782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1233/2468] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:39.763158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:39.764891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:39.785313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c9f61-f706-4931-9181-196e2bb9d454 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:39.790226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:39.790841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:39.791191 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:39.791350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] PATH_INFO: `/v3/projects/d2a8900a1b484daa9d3c42d3c619efc7` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:39.792535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a8900a1b484daa9d3c42d3c619efc7)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:39.794801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:39.844693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a8900a1b484daa9d3c42d3c619efc7', 'request_id': 'req-548600a5-b0c3-4bcc-830d-c6c083c6e667'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.852522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a8900a1b484daa9d3c42d3c619efc7', 'request_id': 'req-548600a5-b0c3-4bcc-830d-c6c083c6e667'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.858250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a8900a1b484daa9d3c42d3c619efc7', 'request_id': 'req-548600a5-b0c3-4bcc-830d-c6c083c6e667'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.875005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] The token cache is being invalidate because project d2a8900a1b484daa9d3c42d3c619efc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:05:39.875295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.875952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:39.878982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1236/2469] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:05:39 2022] DELETE /identity/v3/projects/d2a8900a1b484daa9d3c42d3c619efc7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:39.882555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:39.894085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523fb06-e105-42e1-aaa0-66f1006efffb service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:39.915300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:39.959743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:39.960793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:39.961068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:39.961334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] PATH_INFO: `/v3/users/9c6a97513a064f279535c3fb7b491531` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:39.962751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c6a97513a064f279535c3fb7b491531)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:39.964856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.012477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c6a97513a064f279535c3fb7b491531'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.013154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c6a97513a064f279535c3fb7b491531'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.021449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c6a97513a064f279535c3fb7b491531'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.035221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c6a97513a064f279535c3fb7b491531'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.042420 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-703763e9-6e96-4eee-bd6d-ddc4de899ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c6a97513a064f279535c3fb7b491531'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.050780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1234/2470] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:05:39 2022] DELETE /identity/v3/users/9c6a97513a064f279535c3fb7b491531 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:40.325380 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:40.326823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:40.340333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548600a5-b0c3-4bcc-830d-c6c083c6e667 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:40.373323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:40.402079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:40.402656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.402929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.403192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] PATH_INFO: `/v3/projects/72c6a91ca74c484badfa941b135c848c` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.404305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72c6a91ca74c484badfa941b135c848c)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:40.406251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.452419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c6a91ca74c484badfa941b135c848c', 'request_id': 'req-8afed92e-2eea-49ae-85e7-13fd02935a4f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.459403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c6a91ca74c484badfa941b135c848c', 'request_id': 'req-8afed92e-2eea-49ae-85e7-13fd02935a4f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.465393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c6a91ca74c484badfa941b135c848c', 'request_id': 'req-8afed92e-2eea-49ae-85e7-13fd02935a4f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.476121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] The token cache is being invalidate because project 72c6a91ca74c484badfa941b135c848c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:05:40.476403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.477089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8afed92e-2eea-49ae-85e7-13fd02935a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:40.478553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1237/2471] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:05:40 2022] DELETE /identity/v3/projects/72c6a91ca74c484badfa941b135c848c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:40.483575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4b4f-5821-4aab-a1c8-6b2d4d1000fb None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.484076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4b4f-5821-4aab-a1c8-6b2d4d1000fb None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.484175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4b4f-5821-4aab-a1c8-6b2d4d1000fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.504576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-58af4b4f-5821-4aab-a1c8-6b2d4d1000fb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:40.505821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-58af4b4f-5821-4aab-a1c8-6b2d4d1000fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:40.564327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1235/2472] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:40.572206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774872c-8bb3-4196-91dc-525f7304799b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.572483 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774872c-8bb3-4196-91dc-525f7304799b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.572781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774872c-8bb3-4196-91dc-525f7304799b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.592742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a774872c-8bb3-4196-91dc-525f7304799b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:40.594047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a774872c-8bb3-4196-91dc-525f7304799b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:40.604868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1238/2473] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:40.608860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af4b4f-5821-4aab-a1c8-6b2d4d1000fb None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:40.619527 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af4b4f-5821-4aab-a1c8-6b2d4d1000fb None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:40.624168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:40.624794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.625075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.625346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.626021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:05:40.626323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:40.627961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.639359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1236/2474] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:05:40.643712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774872c-8bb3-4196-91dc-525f7304799b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:40.653078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774872c-8bb3-4196-91dc-525f7304799b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:40.657882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:40.658976 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.659265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.659496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.660122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:40.662045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.688666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1239/2475] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:05:40 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:05:40.692941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:40.702947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9550107-4864-43d9-8f46-9f7994ece58a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:40.731645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:40.760959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:40.761549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.761820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.762083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.762708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:40.764622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.787846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1237/2476] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:05:40 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:40.792010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:40.802799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c95946-d902-4592-acd1-3a2acda6ebc2 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:40.807478 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:40.808066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.808344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.808613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.809289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:40.810890 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.820934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1240/2477] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:40.825017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:40.834950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37494029-b8b7-4fee-b2f8-acb6d1552268 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:40.839765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c18ee-1922-4b5f-b388-76cd8faf11e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:40.840343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c18ee-1922-4b5f-b388-76cd8faf11e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.840617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c18ee-1922-4b5f-b388-76cd8faf11e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.840911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c18ee-1922-4b5f-b388-76cd8faf11e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.841541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c18ee-1922-4b5f-b388-76cd8faf11e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:40.843136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c18ee-1922-4b5f-b388-76cd8faf11e6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.853373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1238/2478] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:40.857375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:40.867985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c121d34-dd22-41d5-b27d-f71c25f639b6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:40.872873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:40.873514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.873791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.874061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] PATH_INFO: `/v3/projects/1b8950d817bf4860a2e68750cd57ba87/users/63da32cb50aa4a79854186a620c9dbc0/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.888535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8950d817bf4860a2e68750cd57ba87, user_id=63da32cb50aa4a79854186a620c9dbc0, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:40.891238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:40.904260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1241/2479] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:40 2022] PUT /identity/v3/projects/1b8950d817bf4860a2e68750cd57ba87/users/63da32cb50aa4a79854186a620c9dbc0/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:40.916158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0be47-2c65-47f8-9ff2-4e22f8081dd8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:40.916464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0be47-2c65-47f8-9ff2-4e22f8081dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:40.916770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0be47-2c65-47f8-9ff2-4e22f8081dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:40.948896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-96f0be47-2c65-47f8-9ff2-4e22f8081dd8 None None] MFA Rules not processed for user `63da32cb50aa4a79854186a620c9dbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:40.950126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-96f0be47-2c65-47f8-9ff2-4e22f8081dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:41.015381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1239/2480] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:41.020410 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:41.089155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb77d-7d5b-4481-b917-07335c483d4e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:41.093891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:41.094503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.094782 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.095051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.096277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:41.098281 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:41.103494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:41.133841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1242/2481] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:05:41 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:41.137876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0be47-2c65-47f8-9ff2-4e22f8081dd8 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:41.150977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0be47-2c65-47f8-9ff2-4e22f8081dd8 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:41.216688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:41.217324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.217599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.217865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.218496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:41.221243 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:41.246683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1240/2482] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:05:41 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:41.249408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:41.259863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdbf04-6a87-4ab9-97b5-54c404ed9596 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:41.264623 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:41.265279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.265556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.265824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.266465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:41.268644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:41.278665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1243/2483] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:05:41.282681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:41.293590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65173c4d-8c42-4b1d-b483-cd3e124971e9 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:41.298195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:41.299376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.299660 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.299937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] PATH_INFO: `/v3/projects/8803f850394147b79248af5528533b95/users/c252783cc1d24ba6bd3cb0fcf0b87f0f/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.316447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8803f850394147b79248af5528533b95, user_id=c252783cc1d24ba6bd3cb0fcf0b87f0f, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:41.319180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:41.332631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1241/2484] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:41 2022] PUT /identity/v3/projects/8803f850394147b79248af5528533b95/users/c252783cc1d24ba6bd3cb0fcf0b87f0f/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:41.336647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:41.347355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dd46a-706c-46ea-8e9f-649b40cdb42f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:41.402545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343173d-4ef7-453d-881b-232d6e138119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:41.403139 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343173d-4ef7-453d-881b-232d6e138119 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.403416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343173d-4ef7-453d-881b-232d6e138119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.403684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343173d-4ef7-453d-881b-232d6e138119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.404312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343173d-4ef7-453d-881b-232d6e138119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:41.405938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343173d-4ef7-453d-881b-232d6e138119 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:41.415475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1244/2485] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:41.419650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:41.426413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:41.430323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493618ec-7f41-47a0-8974-d5b63b947d87 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:41.435802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:41.436446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.436743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.437022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] PATH_INFO: `/v3/projects/8803f850394147b79248af5528533b95/users/c252783cc1d24ba6bd3cb0fcf0b87f0f/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.438919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8803f850394147b79248af5528533b95, user_id=c252783cc1d24ba6bd3cb0fcf0b87f0f, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:41.446215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:41.459381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1242/2486] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:41 2022] PUT /identity/v3/projects/8803f850394147b79248af5528533b95/users/c252783cc1d24ba6bd3cb0fcf0b87f0f/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:05:41.469823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62aff9d-6499-408d-bad8-e6f379433432 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.470105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62aff9d-6499-408d-bad8-e6f379433432 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.470378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62aff9d-6499-408d-bad8-e6f379433432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.497924 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f62aff9d-6499-408d-bad8-e6f379433432 None None] MFA Rules not processed for user `c252783cc1d24ba6bd3cb0fcf0b87f0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:41.499130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f62aff9d-6499-408d-bad8-e6f379433432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:41.566839 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1245/2487] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:41.578316 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:41.580747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:41.637542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23aad-3f68-42ae-baab-6416c4b11e82 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:41.641267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5d234-adc9-4ffb-b79c-3354176753f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:41.641863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d234-adc9-4ffb-b79c-3354176753f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:41.642133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d234-adc9-4ffb-b79c-3354176753f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:41.642416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5d234-adc9-4ffb-b79c-3354176753f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:41.652419 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5d234-adc9-4ffb-b79c-3354176753f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:41.653976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a5d234-adc9-4ffb-b79c-3354176753f0 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:41.664119 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1243/2488] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:42.088825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62aff9d-6499-408d-bad8-e6f379433432 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:42.150179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:42.156858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62aff9d-6499-408d-bad8-e6f379433432 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:42.163303 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:42.163953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:42.164226 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:42.164493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] PATH_INFO: `/v3/users/63da32cb50aa4a79854186a620c9dbc0` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:42.165653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63da32cb50aa4a79854186a620c9dbc0)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:42.167558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:42.213209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63da32cb50aa4a79854186a620c9dbc0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.220126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63da32cb50aa4a79854186a620c9dbc0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.232435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63da32cb50aa4a79854186a620c9dbc0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.239733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63da32cb50aa4a79854186a620c9dbc0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.240367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63da32cb50aa4a79854186a620c9dbc0'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.246119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1246/2489] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:05:42 2022] DELETE /identity/v3/users/63da32cb50aa4a79854186a620c9dbc0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:42.250819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:42.251096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:42.251412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:42.271672 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:42.273007 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:42.337784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1244/2490] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:42.348377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:42.349806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:42.420386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ebc7e6-2556-447a-83c8-7fe92cae1049 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:42.424799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:42.425822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:42.426092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:42.426356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:42.435521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:42.437001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:42.446287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1247/2491] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:42.715623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:42.717142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:42.729568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b7e6-5fa0-49e3-a359-9689ed04bb96 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:42.734108 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:42.734726 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:42.734998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:42.735263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] PATH_INFO: `/v3/projects/1b8950d817bf4860a2e68750cd57ba87` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:42.736370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b8950d817bf4860a2e68750cd57ba87)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:42.738332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:42.785022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b8950d817bf4860a2e68750cd57ba87', 'request_id': 'req-3ab1a923-640d-45b3-bfd4-72991bc90d8a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.792558 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b8950d817bf4860a2e68750cd57ba87', 'request_id': 'req-3ab1a923-640d-45b3-bfd4-72991bc90d8a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.798442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b8950d817bf4860a2e68750cd57ba87', 'request_id': 'req-3ab1a923-640d-45b3-bfd4-72991bc90d8a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.810276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] The token cache is being invalidate because project 1b8950d817bf4860a2e68750cd57ba87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:05:42.810564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.811206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.812686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1245/2492] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:05:42 2022] DELETE /identity/v3/projects/1b8950d817bf4860a2e68750cd57ba87 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:42.817328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:42.826763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3e4e9-0086-40b9-8eb6-c22b923ae3b7 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:42.883246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:42.883895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:42.884147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:42.884412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] PATH_INFO: `/v3/users/c252783cc1d24ba6bd3cb0fcf0b87f0f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:42.885566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c252783cc1d24ba6bd3cb0fcf0b87f0f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:42.887466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:42.908476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:42.936101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c252783cc1d24ba6bd3cb0fcf0b87f0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.944318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c252783cc1d24ba6bd3cb0fcf0b87f0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.957010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c252783cc1d24ba6bd3cb0fcf0b87f0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.963852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c252783cc1d24ba6bd3cb0fcf0b87f0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.964486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-959af75f-1105-459f-92bd-62dc00ba524b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c252783cc1d24ba6bd3cb0fcf0b87f0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:42.970251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1248/2493] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:05:42 2022] DELETE /identity/v3/users/c252783cc1d24ba6bd3cb0fcf0b87f0f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:05:43.249330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:43.250880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:43.269220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1a923-640d-45b3-bfd4-72991bc90d8a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:43.283284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:43.331333 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:43.331961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.332236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.332505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] PATH_INFO: `/v3/projects/8803f850394147b79248af5528533b95` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.333654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8803f850394147b79248af5528533b95)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:43.335575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:43.382624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8803f850394147b79248af5528533b95', 'request_id': 'req-c6abb47e-4e18-4333-b4ec-d248221d02fe'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:43.390115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8803f850394147b79248af5528533b95', 'request_id': 'req-c6abb47e-4e18-4333-b4ec-d248221d02fe'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:43.395719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8803f850394147b79248af5528533b95', 'request_id': 'req-c6abb47e-4e18-4333-b4ec-d248221d02fe'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:43.406543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] The token cache is being invalidate because project 8803f850394147b79248af5528533b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:05:43.406826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:43.407455 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c6abb47e-4e18-4333-b4ec-d248221d02fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:43.409436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1246/2494] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:05:43 2022] DELETE /identity/v3/projects/8803f850394147b79248af5528533b95 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:43.414412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a907f-28f5-40de-a5f1-5e87850da61a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.414889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a907f-28f5-40de-a5f1-5e87850da61a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.414993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8a907f-28f5-40de-a5f1-5e87850da61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.435044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-af8a907f-28f5-40de-a5f1-5e87850da61a None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:43.436242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-af8a907f-28f5-40de-a5f1-5e87850da61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:43.499977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1249/2495] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:43.509925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ca49d-b496-4c17-bc73-e6c935dd201f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.510205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ca49d-b496-4c17-bc73-e6c935dd201f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.510474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ca49d-b496-4c17-bc73-e6c935dd201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.530011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e68ca49d-b496-4c17-bc73-e6c935dd201f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:43.531311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e68ca49d-b496-4c17-bc73-e6c935dd201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:43.540354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1247/2496] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:43.544829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a907f-28f5-40de-a5f1-5e87850da61a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:43.555174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8a907f-28f5-40de-a5f1-5e87850da61a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:43.559813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:43.560417 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.560697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.561178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.561829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:05:43.562130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:43.563751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:43.575939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1250/2497] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:05:43.580231 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ca49d-b496-4c17-bc73-e6c935dd201f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:43.589951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ca49d-b496-4c17-bc73-e6c935dd201f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:43.594834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:43.595946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.596229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.596502 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.597183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:43.599233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:43.627181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1248/2498] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:05:43 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:43.631392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:43.641324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63e1ae-db8d-4d62-a190-0752af130b6d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:43.697619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:43.698248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.698523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.698792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.699415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:43.701377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:43.723019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1251/2499] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:05:43 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:43.727192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:43.738312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f651fba-efce-4263-a452-cb0fd4b6c42e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:43.743057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:43.743768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.744050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.744322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.745006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:43.746615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:43.756780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1249/2500] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:43.760610 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:43.770451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc046-6911-4350-83f2-66e247aa69df admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:43.775430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a78094-40cb-43fe-9497-5fcfc15be5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:43.776033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78094-40cb-43fe-9497-5fcfc15be5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.776339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78094-40cb-43fe-9497-5fcfc15be5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.776611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a78094-40cb-43fe-9497-5fcfc15be5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.777295 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a78094-40cb-43fe-9497-5fcfc15be5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:43.778889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a78094-40cb-43fe-9497-5fcfc15be5e1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:43.789974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1252/2501] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:05:43.794114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:43.805046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348648c7-3e2c-47a9-b0a9-79f0e1a020d3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:43.809736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:43.810374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.810654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.810929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] PATH_INFO: `/v3/projects/ab0098597c604fba8d951162a93cd7f8/users/5f88da39102d49fe9916c304a90c1a00/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.825519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0098597c604fba8d951162a93cd7f8, user_id=5f88da39102d49fe9916c304a90c1a00, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:43.828405 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:43.842105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1250/2502] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:43 2022] PUT /identity/v3/projects/ab0098597c604fba8d951162a93cd7f8/users/5f88da39102d49fe9916c304a90c1a00/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:43.853578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff73089-30fc-4c45-86ed-da72fc0f9d37 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:43.853861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff73089-30fc-4c45-86ed-da72fc0f9d37 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:43.854130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff73089-30fc-4c45-86ed-da72fc0f9d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:43.862427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:43.884797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7ff73089-30fc-4c45-86ed-da72fc0f9d37 None None] MFA Rules not processed for user `5f88da39102d49fe9916c304a90c1a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:43.886008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7ff73089-30fc-4c45-86ed-da72fc0f9d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:43.949074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1253/2503] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:43.953746 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.004373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:44.036800 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266e5f8-ba5f-45a0-80c9-074e20d8848c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.047309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.047735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.048024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.048299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.049311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.051441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.175821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1251/2504] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:05:43 2022] POST /identity/v3/projects => generated 355 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:05:44.179957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff73089-30fc-4c45-86ed-da72fc0f9d37 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.191245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff73089-30fc-4c45-86ed-da72fc0f9d37 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.252925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.253538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.253817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.254091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.254733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.256699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.280222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1254/2505] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:05:44 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:05:44.284891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.295505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df74d8-8f77-466f-8c1e-44a9ef0a4922 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.300545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.301175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.301456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.301729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.302371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.303992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.315747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1252/2506] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:44.319933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.330966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a1c14-15fd-4ad5-9a33-9b8d371f7c0a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.335457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.336101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.336374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.336641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] PATH_INFO: `/v3/projects/ecee1d2e28824cccb3092c0aba26723f/users/c6aaa58cacf04e9cb3314d2cb4e0c946/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.351986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecee1d2e28824cccb3092c0aba26723f, user_id=c6aaa58cacf04e9cb3314d2cb4e0c946, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.354641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.367304 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1255/2507] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:44 2022] PUT /identity/v3/projects/ecee1d2e28824cccb3092c0aba26723f/users/c6aaa58cacf04e9cb3314d2cb4e0c946/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:44.371257 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.382944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5893a-e1fe-41eb-b3f8-31c154c8a4fa admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.439641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337ef64-1c25-4bc1-9b43-e43023c1eb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.440255 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337ef64-1c25-4bc1-9b43-e43023c1eb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.440657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337ef64-1c25-4bc1-9b43-e43023c1eb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.440971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337ef64-1c25-4bc1-9b43-e43023c1eb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.441613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337ef64-1c25-4bc1-9b43-e43023c1eb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.443227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337ef64-1c25-4bc1-9b43-e43023c1eb20 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.456714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1253/2508] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:05:44.460914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.471343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9a9c2-0fad-4d6c-93eb-652539652c82 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.476955 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.477611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.477889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.478163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] PATH_INFO: `/v3/projects/ecee1d2e28824cccb3092c0aba26723f/users/c6aaa58cacf04e9cb3314d2cb4e0c946/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.480149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecee1d2e28824cccb3092c0aba26723f, user_id=c6aaa58cacf04e9cb3314d2cb4e0c946, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.482893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.493644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1256/2509] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:05:44 2022] PUT /identity/v3/projects/ecee1d2e28824cccb3092c0aba26723f/users/c6aaa58cacf04e9cb3314d2cb4e0c946/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:05:44.504541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aaeb6-8f76-49e6-873a-e713cebe542e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.504856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aaeb6-8f76-49e6-873a-e713cebe542e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.505130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aaeb6-8f76-49e6-873a-e713cebe542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.536358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-204aaeb6-8f76-49e6-873a-e713cebe542e None None] MFA Rules not processed for user `c6aaa58cacf04e9cb3314d2cb4e0c946`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:44.537738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-204aaeb6-8f76-49e6-873a-e713cebe542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:44.615629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1254/2510] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:05:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:44.623275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.625112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:44.677015 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:44.690558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebcc18-305d-4f84-b2a4-a10f077c1193 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.694310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.694927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.695195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.695460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.704959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.706425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.716239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1257/2511] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:44.801432 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204aaeb6-8f76-49e6-873a-e713cebe542e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:44.811239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204aaeb6-8f76-49e6-873a-e713cebe542e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:44.815512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12142a65-de6f-47e4-8b38-f653f0c33934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:44.816117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12142a65-de6f-47e4-8b38-f653f0c33934 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:44.816397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12142a65-de6f-47e4-8b38-f653f0c33934 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:44.816669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12142a65-de6f-47e4-8b38-f653f0c33934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:44.856073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:44.877535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12142a65-de6f-47e4-8b38-f653f0c33934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:44.879329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12142a65-de6f-47e4-8b38-f653f0c33934 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:44.888144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1255/2512] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:45.970216 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:45.971999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:46.033141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb8457-c1a6-4cad-9986-408a0382adba service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:46.036791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:46.037392 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:46.037663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:46.037928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:46.039742 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:46.112207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:46.113681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:46.123801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1258/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:48.675533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:48.676124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:48.676124 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:48.704699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:05:48.706636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:48.793301 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1256/2514] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 16:05:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:05:48.799988 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:48.801638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:48.880321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9222fb-5bff-4e3f-b06a-b1571c3be0c7 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:48.885337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:48.886117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:48.886209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:48.886468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:48.895106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:48.896593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:48.904900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1259/2515] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:05:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:05:52.322767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:52.324324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:52.337658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6949c7-1e4e-474f-82bd-4d2b19b265bf None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:52.341473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:52.342064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:52.342340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:52.342609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:52.344343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:52.411709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:52.413150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:52.423173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1257/2516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:05:58.902591 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=86 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:59.165162 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:59.166857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:59.186073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71354b-d576-468b-9b94-ba35a97ddb11 service swift] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:59.193156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:59.194469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:59.194932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:59.195283 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] PATH_INFO: `/v3/users/950e06d7bfff4259972ae1312e8f913a` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:59.196567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=950e06d7bfff4259972ae1312e8f913a)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:59.198813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:59.260657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950e06d7bfff4259972ae1312e8f913a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.270452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950e06d7bfff4259972ae1312e8f913a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.289837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950e06d7bfff4259972ae1312e8f913a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.296865 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950e06d7bfff4259972ae1312e8f913a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.297562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950e06d7bfff4259972ae1312e8f913a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.305512 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1260/2517] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:05:59 2022] DELETE /identity/v3/users/950e06d7bfff4259972ae1312e8f913a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:59.495389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:59.511430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae327ac-6ad7-4349-b55a-f659805df74b service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:59.588614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:59.589287 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:59.589571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:59.589841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] PATH_INFO: `/v3/projects/60b016d4443142d4bb81dae536748cf1` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:59.591167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b016d4443142d4bb81dae536748cf1)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:59.593477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:59.617332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:59.645143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b016d4443142d4bb81dae536748cf1', 'request_id': 'req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.656247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b016d4443142d4bb81dae536748cf1', 'request_id': 'req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.674142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b016d4443142d4bb81dae536748cf1', 'request_id': 'req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.688039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] The token cache is being invalidate because project 60b016d4443142d4bb81dae536748cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:05:59.688326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.689493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.690766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1258/2518] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:05:59 2022] DELETE /identity/v3/projects/60b016d4443142d4bb81dae536748cf1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:05:59.694950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:05:59.696444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:05:59.710445 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605adc9-b0c8-4c04-a38b-bd94f8eb30e4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:05:59.768996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:05:59.769616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:05:59.769889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:05:59.770154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] PATH_INFO: `/v3/users/28e6345f277e45a7b9a3636737c08b29` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:05:59.771305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e6345f277e45a7b9a3636737c08b29)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:05:59.773306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:05:59.819776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e6345f277e45a7b9a3636737c08b29'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.826984 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e6345f277e45a7b9a3636737c08b29'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.839939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e6345f277e45a7b9a3636737c08b29'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.841896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:05:59.849213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e6345f277e45a7b9a3636737c08b29'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.849937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e6345f277e45a7b9a3636737c08b29'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:05:59.856122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1261/2519] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:05:59 2022] DELETE /identity/v3/users/28e6345f277e45a7b9a3636737c08b29 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:00.030166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:00.031671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:00.047773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdc89a-dcb3-4132-ba5c-291d94bc5e0f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:00.109662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:00.110318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:00.110722 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:00.111008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] PATH_INFO: `/v3/projects/85c573761eb54bd5bfe7e8a9a0088041` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:00.112139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c573761eb54bd5bfe7e8a9a0088041)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:00.114128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:00.166653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c573761eb54bd5bfe7e8a9a0088041', 'request_id': 'req-59bf561c-d41b-444e-99d5-c78413a462eb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.174630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c573761eb54bd5bfe7e8a9a0088041', 'request_id': 'req-59bf561c-d41b-444e-99d5-c78413a462eb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.180966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c573761eb54bd5bfe7e8a9a0088041', 'request_id': 'req-59bf561c-d41b-444e-99d5-c78413a462eb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.193751 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] The token cache is being invalidate because project 85c573761eb54bd5bfe7e8a9a0088041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:00.194302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.195020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-59bf561c-d41b-444e-99d5-c78413a462eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.196771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1259/2520] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:00 2022] DELETE /identity/v3/projects/85c573761eb54bd5bfe7e8a9a0088041 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:00.399459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:00.401065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:00.415233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c37af-ac5f-487c-a0e1-e79bd9bac1de admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:00.477950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:00.478496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:00.478775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:00.479044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] PATH_INFO: `/v3/users/5f88da39102d49fe9916c304a90c1a00` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:00.480165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f88da39102d49fe9916c304a90c1a00)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:00.482123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:00.527798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f88da39102d49fe9916c304a90c1a00'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.536194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f88da39102d49fe9916c304a90c1a00'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.548473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f88da39102d49fe9916c304a90c1a00'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.555272 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f88da39102d49fe9916c304a90c1a00'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.555939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f88da39102d49fe9916c304a90c1a00'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:00.561712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1262/2521] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:00 2022] DELETE /identity/v3/users/5f88da39102d49fe9916c304a90c1a00 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:00.566303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:00.566796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:00.566895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:00.587014 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:00.588207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:00.625670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:00.656112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1260/2522] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:00.665773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:00.667160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:00.728471 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c5c20-243f-43f7-b77e-3aa3bc35e93e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:00.732103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:00.732738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:00.733038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:00.733308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:00.743116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:00.744535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:00.754566 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1263/2523] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:01.023068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:01.024506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:01.038389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8e219-0ba4-4525-9fe3-2ae9a73ba8a5 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:01.043054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:01.043682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.043956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.044222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] PATH_INFO: `/v3/projects/ab0098597c604fba8d951162a93cd7f8` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.045375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0098597c604fba8d951162a93cd7f8)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:01.047454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:01.094631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0098597c604fba8d951162a93cd7f8', 'request_id': 'req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.102100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0098597c604fba8d951162a93cd7f8', 'request_id': 'req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.107748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0098597c604fba8d951162a93cd7f8', 'request_id': 'req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.119138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] The token cache is being invalidate because project ab0098597c604fba8d951162a93cd7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:01.119423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.120048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.121538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1261/2524] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:01 2022] DELETE /identity/v3/projects/ab0098597c604fba8d951162a93cd7f8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:01.125639 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:01.135601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae1e81-d24b-4cc3-9646-3613969b67a8 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:01.144452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:01.194073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:01.194702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.194987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.195254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] PATH_INFO: `/v3/users/c6aaa58cacf04e9cb3314d2cb4e0c946` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.196372 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6aaa58cacf04e9cb3314d2cb4e0c946)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:01.198379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:01.243839 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6aaa58cacf04e9cb3314d2cb4e0c946'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.251117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6aaa58cacf04e9cb3314d2cb4e0c946'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.263223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6aaa58cacf04e9cb3314d2cb4e0c946'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.270872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6aaa58cacf04e9cb3314d2cb4e0c946'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.271482 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b6c4461f-4a40-4980-858e-d1b8467392f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6aaa58cacf04e9cb3314d2cb4e0c946'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.277267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1264/2525] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:01 2022] DELETE /identity/v3/users/c6aaa58cacf04e9cb3314d2cb4e0c946 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:01.546612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:01.548142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:01.561966 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564468ff-ddc0-4828-883f-7cc3ebdcb4cf admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:01.627982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:01.628636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.628913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.629185 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] PATH_INFO: `/v3/projects/ecee1d2e28824cccb3092c0aba26723f` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.630299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecee1d2e28824cccb3092c0aba26723f)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:01.632210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:01.678860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecee1d2e28824cccb3092c0aba26723f', 'request_id': 'req-da1964d7-139e-4675-8e30-0264b8b3d253'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.685211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecee1d2e28824cccb3092c0aba26723f', 'request_id': 'req-da1964d7-139e-4675-8e30-0264b8b3d253'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.690308 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:01.691521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecee1d2e28824cccb3092c0aba26723f', 'request_id': 'req-da1964d7-139e-4675-8e30-0264b8b3d253'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.705426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] The token cache is being invalidate because project ecee1d2e28824cccb3092c0aba26723f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:01.705728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.706398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da1964d7-139e-4675-8e30-0264b8b3d253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:01.707865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1262/2526] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:01 2022] DELETE /identity/v3/projects/ecee1d2e28824cccb3092c0aba26723f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:01.713196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe2726-3de2-4da5-ba5f-b6288ea9f89c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.713773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe2726-3de2-4da5-ba5f-b6288ea9f89c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.713773 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe2726-3de2-4da5-ba5f-b6288ea9f89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.734199 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-85fe2726-3de2-4da5-ba5f-b6288ea9f89c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:01.735386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-85fe2726-3de2-4da5-ba5f-b6288ea9f89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:01.795124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1265/2527] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:01.803102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba83bc-2a7a-49d6-9136-7d8b4ac50963 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.803383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba83bc-2a7a-49d6-9136-7d8b4ac50963 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.803648 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba83bc-2a7a-49d6-9136-7d8b4ac50963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.823488 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-67ba83bc-2a7a-49d6-9136-7d8b4ac50963 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:01.824671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-67ba83bc-2a7a-49d6-9136-7d8b4ac50963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:01.833423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1263/2528] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:01.837771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe2726-3de2-4da5-ba5f-b6288ea9f89c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:01.847405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe2726-3de2-4da5-ba5f-b6288ea9f89c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:01.851937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:01.852525 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.852827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.853094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.853731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:06:01.854027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:01.855620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:01.866735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1266/2529] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:06:01.870948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba83bc-2a7a-49d6-9136-7d8b4ac50963 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:01.880539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba83bc-2a7a-49d6-9136-7d8b4ac50963 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:01.885324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:01.886033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.886312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.886581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.887245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:01.889250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:01.916592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1264/2530] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:01 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:01.920613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:01.930929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db54370-b94e-4aeb-bb70-5a11d9b5b3f1 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:01.948660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:01.988544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:01.989176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:01.989449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:01.989715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:01.990330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:01.992237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.015975 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1267/2531] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:01 2022] POST /identity/v3/users => generated 304 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:02.020114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.030278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7178-98d0-4a9a-938e-60626ffc533c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.036057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.036626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.036917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.037206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.037845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.039440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.050270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1265/2532] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:02.054267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.064660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258e14-0731-4f1a-891b-0b7ff93c2b44 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.069243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549033d-ab69-4767-aca3-e47f68097b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.069830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549033d-ab69-4767-aca3-e47f68097b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.070104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549033d-ab69-4767-aca3-e47f68097b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.070365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549033d-ab69-4767-aca3-e47f68097b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.070995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549033d-ab69-4767-aca3-e47f68097b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.072585 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549033d-ab69-4767-aca3-e47f68097b29 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.083173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1268/2533] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:02.087647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.097910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531241e-b033-4610-bdf7-b1140df324d8 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.102588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.103223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.103496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.103761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] PATH_INFO: `/v3/projects/2f768fc195c64bdf83f7267d502e581d/users/5fd2009cba974f4cadbadf33d0ad8024/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.119067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f768fc195c64bdf83f7267d502e581d, user_id=5fd2009cba974f4cadbadf33d0ad8024, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.121840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.135334 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1266/2534] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:02 2022] PUT /identity/v3/projects/2f768fc195c64bdf83f7267d502e581d/users/5fd2009cba974f4cadbadf33d0ad8024/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:02.145613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07176-e66c-44c8-b243-d36ad8aabe57 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.145903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07176-e66c-44c8-b243-d36ad8aabe57 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.146181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07176-e66c-44c8-b243-d36ad8aabe57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.173708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-28b07176-e66c-44c8-b243-d36ad8aabe57 None None] MFA Rules not processed for user `5fd2009cba974f4cadbadf33d0ad8024`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:02.175110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-28b07176-e66c-44c8-b243-d36ad8aabe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:02.241242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1269/2535] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:06:02.245910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.306415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5438a4e-b443-472f-abb7-1bd24aa9c3dd admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.310913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.311508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.311780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.312047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.312672 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.315330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.342355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1267/2536] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:02 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:02.346717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b07176-e66c-44c8-b243-d36ad8aabe57 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.357360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b07176-e66c-44c8-b243-d36ad8aabe57 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.414182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.414779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.415057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.415327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.415944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.417954 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.438586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1270/2537] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:02 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:02.443357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.453697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16960b1-c72d-4956-bd57-7c450ff2ca57 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.459084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.459678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.459956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.460227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.460900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.462777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.473435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1268/2538] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:02.477549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.487456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1b854-3bec-4ff9-9171-fdc9ee5e8139 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.492168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.492842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.493123 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.493397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] PATH_INFO: `/v3/projects/2ba06aa94c424768a6d397d5f070f844/users/9975584cb41e426e83bf2fd1d1fac8e7/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.508258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba06aa94c424768a6d397d5f070f844, user_id=9975584cb41e426e83bf2fd1d1fac8e7, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.510950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.513125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:02.524635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1271/2539] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:02 2022] PUT /identity/v3/projects/2ba06aa94c424768a6d397d5f070f844/users/9975584cb41e426e83bf2fd1d1fac8e7/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:02.528652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.539493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a157c4-31c1-4f31-9a45-7b6926242a7b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.595775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e53575-9712-47de-ba77-e75243c2ff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.596376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e53575-9712-47de-ba77-e75243c2ff16 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.596657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e53575-9712-47de-ba77-e75243c2ff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.596958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e53575-9712-47de-ba77-e75243c2ff16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.597596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e53575-9712-47de-ba77-e75243c2ff16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.599205 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e53575-9712-47de-ba77-e75243c2ff16 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.601360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:02.613094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1269/2540] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:02.615111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.625395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba908-0262-410a-a25d-eb80581cc624 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.630168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.630801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.631072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.631336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] PATH_INFO: `/v3/projects/2ba06aa94c424768a6d397d5f070f844/users/9975584cb41e426e83bf2fd1d1fac8e7/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.633250 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba06aa94c424768a6d397d5f070f844, user_id=9975584cb41e426e83bf2fd1d1fac8e7, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.635854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.645823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1272/2541] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:02 2022] PUT /identity/v3/projects/2ba06aa94c424768a6d397d5f070f844/users/9975584cb41e426e83bf2fd1d1fac8e7/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:02.656875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6c5d-5507-478c-b1b3-517d41964abc None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.657349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6c5d-5507-478c-b1b3-517d41964abc None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.657974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d6c5d-5507-478c-b1b3-517d41964abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.686765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-852d6c5d-5507-478c-b1b3-517d41964abc None None] MFA Rules not processed for user `9975584cb41e426e83bf2fd1d1fac8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:02.687990 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-852d6c5d-5507-478c-b1b3-517d41964abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:02.754217 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1270/2542] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:02.765597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:02.768194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:02.826313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9669c-7560-4ab4-a24a-726bb98f3f30 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:02.830907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:02.831510 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:02.831788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:02.832070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:02.841567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:02.843260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:02.858061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1273/2543] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:02 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:03.005638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6c5d-5507-478c-b1b3-517d41964abc None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:03.014974 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d6c5d-5507-478c-b1b3-517d41964abc None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:03.019063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84945f7-74b5-4939-a5c4-47160cfb7299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:03.019645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84945f7-74b5-4939-a5c4-47160cfb7299 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:03.019916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84945f7-74b5-4939-a5c4-47160cfb7299 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:03.020181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84945f7-74b5-4939-a5c4-47160cfb7299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:03.077126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84945f7-74b5-4939-a5c4-47160cfb7299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:03.078555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84945f7-74b5-4939-a5c4-47160cfb7299 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:03.088066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1271/2544] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:10.429664 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:10.495883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73ed50-92f9-4f59-8ae4-6a8befbb26cb service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:10.500468 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:10.501173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:10.501442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:10.501708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] PATH_INFO: `/v3/users/5fd2009cba974f4cadbadf33d0ad8024` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:10.502825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd2009cba974f4cadbadf33d0ad8024)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:10.504759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:10.553848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd2009cba974f4cadbadf33d0ad8024'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:10.562486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd2009cba974f4cadbadf33d0ad8024'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:10.578168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:10.578988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:10.579957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:10.580900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd2009cba974f4cadbadf33d0ad8024'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:10.588378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd2009cba974f4cadbadf33d0ad8024'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:10.589182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8aff40e6-0351-4bfd-a9e9-3eed76f890d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd2009cba974f4cadbadf33d0ad8024'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:10.595101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1274/2545] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:10 2022] DELETE /identity/v3/users/5fd2009cba974f4cadbadf33d0ad8024 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:10.599463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:10.599615 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] MFA Rules not processed for user `4cc1db8ef8c44763a0e5f62a3404a834`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:10.599910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:10.600275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:10.600639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:10.609048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:10.627685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:10.628935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:10.660603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1272/2546] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:06:10.699038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1275/2547] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:10.706773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:10.708609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:10.747559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:10.776946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6733f-2aab-43a3-a104-e5fcf12a4d77 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:10.781048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:10.782492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:10.783038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:10.783440 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:10.794355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:10.795889 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:10.807856 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1273/2548] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:11.270407 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:11.274113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:11.296834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bbbd7-7ec6-4920-9a19-632c69b6db95 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:11.307388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:11.308090 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:11.308429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:11.308809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] PATH_INFO: `/v3/projects/2f768fc195c64bdf83f7267d502e581d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:11.310312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f768fc195c64bdf83f7267d502e581d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:11.311969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:11.370472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f768fc195c64bdf83f7267d502e581d', 'request_id': 'req-df9235f0-3ff0-4f35-828e-0c497e56ca87'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.378213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f768fc195c64bdf83f7267d502e581d', 'request_id': 'req-df9235f0-3ff0-4f35-828e-0c497e56ca87'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.384603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f768fc195c64bdf83f7267d502e581d', 'request_id': 'req-df9235f0-3ff0-4f35-828e-0c497e56ca87'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.397398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] The token cache is being invalidate because project 2f768fc195c64bdf83f7267d502e581d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:11.397673 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.398449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.400097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1276/2549] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:11 2022] DELETE /identity/v3/projects/2f768fc195c64bdf83f7267d502e581d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:11.404253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:11.414146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384b1a-e227-4544-87ec-de2e2e56d0c2 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:11.483991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:11.484714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:11.485117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:11.485956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] PATH_INFO: `/v3/users/9975584cb41e426e83bf2fd1d1fac8e7` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:11.487105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9975584cb41e426e83bf2fd1d1fac8e7)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:11.489142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:11.539235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9975584cb41e426e83bf2fd1d1fac8e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.539918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9975584cb41e426e83bf2fd1d1fac8e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.548880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9975584cb41e426e83bf2fd1d1fac8e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.562117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9975584cb41e426e83bf2fd1d1fac8e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.569937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-01273499-d2ec-4706-a6c0-18e99c46ecc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9975584cb41e426e83bf2fd1d1fac8e7'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:11.575860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1274/2550] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:11 2022] DELETE /identity/v3/users/9975584cb41e426e83bf2fd1d1fac8e7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:06:11.859852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:11.862069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:11.876452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9235f0-3ff0-4f35-828e-0c497e56ca87 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:11.938899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:11.939559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:11.939843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:11.940108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] PATH_INFO: `/v3/projects/2ba06aa94c424768a6d397d5f070f844` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:11.941289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba06aa94c424768a6d397d5f070f844)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:11.943203 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:11.975057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:11.994072 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba06aa94c424768a6d397d5f070f844', 'request_id': 'req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:12.002741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba06aa94c424768a6d397d5f070f844', 'request_id': 'req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:12.008406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba06aa94c424768a6d397d5f070f844', 'request_id': 'req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:12.020686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] The token cache is being invalidate because project 2ba06aa94c424768a6d397d5f070f844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:12.021003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:12.021653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c38678e8-1fe5-41a3-91cc-d3cedc5e3b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:12.023138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1277/2551] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:11 2022] DELETE /identity/v3/projects/2ba06aa94c424768a6d397d5f070f844 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:12.028122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82088233-e268-47d6-ab47-5e607fec32f4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.028607 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82088233-e268-47d6-ab47-5e607fec32f4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.028708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82088233-e268-47d6-ab47-5e607fec32f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.050241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-82088233-e268-47d6-ab47-5e607fec32f4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:12.051699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-82088233-e268-47d6-ab47-5e607fec32f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:12.097958 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:12.115326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1275/2552] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:12.123087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311b653-f22e-4ff4-b322-6e8626bd5605 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.123245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311b653-f22e-4ff4-b322-6e8626bd5605 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.123527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311b653-f22e-4ff4-b322-6e8626bd5605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.143600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c311b653-f22e-4ff4-b322-6e8626bd5605 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:12.144759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c311b653-f22e-4ff4-b322-6e8626bd5605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:12.153472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1278/2553] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:12.157984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82088233-e268-47d6-ab47-5e607fec32f4 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:12.167691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82088233-e268-47d6-ab47-5e607fec32f4 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:12.172331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:12.173515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.173797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.174062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.174703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:06:12.175004 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:12.176608 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:12.188554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1276/2554] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:06:12.193381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311b653-f22e-4ff4-b322-6e8626bd5605 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:12.202630 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311b653-f22e-4ff4-b322-6e8626bd5605 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:12.207772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:12.208398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.208681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.208986 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.209618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:12.211831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:12.240904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1279/2555] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:12 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:06:12.245100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:12.255327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea2450-2b6c-4e64-ac8a-9cf9bb463820 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:12.312904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:12.313904 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.314179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.314445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.315068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:12.317038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:12.338689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1277/2556] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:12 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:06:12.342825 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:12.354222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79b71f-243f-4dea-be81-bcf689ba45c2 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:12.358857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:12.359443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.359719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.359991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.361219 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:12.362801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:12.372568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1280/2557] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:12.376866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:12.386764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a28cf-ba85-480b-9c9c-63d6329b9519 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:12.392369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119a7fd-53a1-4be3-bf76-9bb0b63bf306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:12.393003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a7fd-53a1-4be3-bf76-9bb0b63bf306 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.393279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a7fd-53a1-4be3-bf76-9bb0b63bf306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.393551 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119a7fd-53a1-4be3-bf76-9bb0b63bf306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.394271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119a7fd-53a1-4be3-bf76-9bb0b63bf306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:12.395872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119a7fd-53a1-4be3-bf76-9bb0b63bf306 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:12.406713 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1278/2558] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:12.411625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:12.422105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59947b2d-24b0-428c-9e24-9383aa8897e8 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:12.427913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:12.428550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.428858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.429130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] PATH_INFO: `/v3/projects/c978b2df3f7b4ce4a4327285d167a4d3/users/330c24fcbf61465d9e573de0ccffc106/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.444082 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c978b2df3f7b4ce4a4327285d167a4d3, user_id=330c24fcbf61465d9e573de0ccffc106, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:12.446780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:12.462818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1281/2559] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:12 2022] PUT /identity/v3/projects/c978b2df3f7b4ce4a4327285d167a4d3/users/330c24fcbf61465d9e573de0ccffc106/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:12.468589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9cdd2-9cac-4c4b-b501-3cd6b60b209e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.468903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9cdd2-9cac-4c4b-b501-3cd6b60b209e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.469173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9cdd2-9cac-4c4b-b501-3cd6b60b209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.491798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-58d9cdd2-9cac-4c4b-b501-3cd6b60b209e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:12.493233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-58d9cdd2-9cac-4c4b-b501-3cd6b60b209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:12.568152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1279/2560] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:12.578129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:12.580708 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:12.651736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c087af4-181f-4d2d-9003-0f16e8825428 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:12.656007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:12.656200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:12.656549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:12.656964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:12.667019 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:12.668546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:12.678726 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1282/2561] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:13.628412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0128b2b-c175-4699-8a6c-b078bde17b1b None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:13.629254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0128b2b-c175-4699-8a6c-b078bde17b1b None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:13.629429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0128b2b-c175-4699-8a6c-b078bde17b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:13.680884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d0128b2b-c175-4699-8a6c-b078bde17b1b None None] MFA Rules not processed for user `330c24fcbf61465d9e573de0ccffc106`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:13.682366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d0128b2b-c175-4699-8a6c-b078bde17b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:13.742688 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:13.775952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1280/2562] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:13.780555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:13.790560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df481d7-e89a-4a95-b370-c50568da5b82 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:13.795793 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:13.796405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:13.796677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:13.796980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:13.797842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:13.799892 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:13.831959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1283/2563] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:13 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:06:13.836701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0128b2b-c175-4699-8a6c-b078bde17b1b None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:13.848572 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0128b2b-c175-4699-8a6c-b078bde17b1b None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:13.918843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:13.919576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:13.919952 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:13.920323 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:13.921141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:13.923719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:13.946478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1281/2564] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:13 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:06:13.950549 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:13.960789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37981855-5a81-4237-9f08-af42fa40d2a7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:13.966039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:13.966723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:13.967065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:13.967413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:13.968115 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:13.970025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:13.980824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1284/2565] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:13.985736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:13.995874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a49323-77a9-49ea-b7c2-199d55270ab3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:14.002232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:14.002902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:14.003183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:14.003456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] PATH_INFO: `/v3/projects/8af03f01df24462a83266b41d0848c03/users/d839885787a5403fab7ed1d990b6230a/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:14.019576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af03f01df24462a83266b41d0848c03, user_id=d839885787a5403fab7ed1d990b6230a, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:14.022351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:14.036684 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1282/2566] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:13 2022] PUT /identity/v3/projects/8af03f01df24462a83266b41d0848c03/users/d839885787a5403fab7ed1d990b6230a/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:14.041005 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:14.051915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c2c16-9657-4f88-a385-a1618e2a52b1 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:14.070902 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:14.112788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f36812-9014-44b7-897a-a97a46f92b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:14.113396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f36812-9014-44b7-897a-a97a46f92b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:14.113670 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f36812-9014-44b7-897a-a97a46f92b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:14.113937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f36812-9014-44b7-897a-a97a46f92b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:14.114578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f36812-9014-44b7-897a-a97a46f92b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:14.116165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f36812-9014-44b7-897a-a97a46f92b25 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:14.126662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1285/2567] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:14.130934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:14.141707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b934289-c86b-4698-8091-e22a9df9d9aa admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:14.146448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:14.147091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:14.147363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:14.147630 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] PATH_INFO: `/v3/projects/8af03f01df24462a83266b41d0848c03/users/d839885787a5403fab7ed1d990b6230a/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:14.149564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8af03f01df24462a83266b41d0848c03, user_id=d839885787a5403fab7ed1d990b6230a, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:14.152202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:14.163091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1283/2568] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:14 2022] PUT /identity/v3/projects/8af03f01df24462a83266b41d0848c03/users/d839885787a5403fab7ed1d990b6230a/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:14.823528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:14.824061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:14.824061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:14.853491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] MFA Rules not processed for user `d839885787a5403fab7ed1d990b6230a`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:14.854710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:14.927508 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1286/2569] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:14.940767 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:14.942223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:15.007157 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd9977-568d-4040-b2e2-82135574487f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:15.010890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:15.011493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:15.011770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:15.012043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:15.069115 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:15.080798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:15.082210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:15.092113 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1284/2570] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:06:16.460016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:16.461612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:16.474243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88e626-8791-4f12-9856-5daa0c8594b8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:16.477775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:16.478366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:16.478637 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:16.478900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:16.487550 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:16.488983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:16.498227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1287/2571] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:25.087581 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:26.494035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=59 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:29.848666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:29.915937 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1335c-c88f-4629-9bec-476277ccb403 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:29.920452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:29.921112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:29.921398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:29.922222 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] PATH_INFO: `/v3/users/330c24fcbf61465d9e573de0ccffc106` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:29.923385 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=330c24fcbf61465d9e573de0ccffc106)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:29.925384 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:29.970758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '330c24fcbf61465d9e573de0ccffc106'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:29.971413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '330c24fcbf61465d9e573de0ccffc106'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:29.978808 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '330c24fcbf61465d9e573de0ccffc106'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:29.991356 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '330c24fcbf61465d9e573de0ccffc106'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:29.998236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '330c24fcbf61465d9e573de0ccffc106'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.004884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1285/2572] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:29 2022] DELETE /identity/v3/users/330c24fcbf61465d9e573de0ccffc106 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:30.177074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:30.189084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7aa62-8ad6-42f5-8b3b-62095272dad9 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:30.248163 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:30.248852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:30.249143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:30.249519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] PATH_INFO: `/v3/projects/c978b2df3f7b4ce4a4327285d167a4d3` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:30.250817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c978b2df3f7b4ce4a4327285d167a4d3)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:30.252862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:30.306493 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c978b2df3f7b4ce4a4327285d167a4d3', 'request_id': 'req-69d16059-b20c-430b-83c1-371151bc9fe1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.314403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c978b2df3f7b4ce4a4327285d167a4d3', 'request_id': 'req-69d16059-b20c-430b-83c1-371151bc9fe1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.320906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c978b2df3f7b4ce4a4327285d167a4d3', 'request_id': 'req-69d16059-b20c-430b-83c1-371151bc9fe1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.332463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] The token cache is being invalidate because project c978b2df3f7b4ce4a4327285d167a4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:30.332808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.333464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.334942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1288/2573] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:30 2022] DELETE /identity/v3/projects/c978b2df3f7b4ce4a4327285d167a4d3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:30.339177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:30.340683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:30.354903 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43e076-8411-445d-a804-f560c719b01c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:30.416196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:30.417446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:30.417935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:30.418260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] PATH_INFO: `/v3/users/d839885787a5403fab7ed1d990b6230a` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:30.419493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d839885787a5403fab7ed1d990b6230a)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:30.421487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:30.445536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:30.477251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd839885787a5403fab7ed1d990b6230a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.477892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd839885787a5403fab7ed1d990b6230a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.485580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd839885787a5403fab7ed1d990b6230a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.498120 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd839885787a5403fab7ed1d990b6230a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.505195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f75679d4-7b13-4ad6-b3a4-f073706b7866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd839885787a5403fab7ed1d990b6230a'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.511034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1286/2574] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:30 2022] DELETE /identity/v3/users/d839885787a5403fab7ed1d990b6230a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:30.855978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:30.858215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:30.871757 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16059-b20c-430b-83c1-371151bc9fe1 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:30.935404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:30.935978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:30.936227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:30.936505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] PATH_INFO: `/v3/projects/8af03f01df24462a83266b41d0848c03` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:30.938382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8af03f01df24462a83266b41d0848c03)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:30.940411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:30.977080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:30.990381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8af03f01df24462a83266b41d0848c03', 'request_id': 'req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:30.997278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8af03f01df24462a83266b41d0848c03', 'request_id': 'req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:31.004036 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8af03f01df24462a83266b41d0848c03', 'request_id': 'req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:31.016948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] The token cache is being invalidate because project 8af03f01df24462a83266b41d0848c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:31.017236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:31.017943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-b0eb33f7-9221-47a4-aa83-1e96eaa707b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:31.019653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1289/2575] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:30 2022] DELETE /identity/v3/projects/8af03f01df24462a83266b41d0848c03 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:31.024657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb53b9-e126-4ad8-bf21-46f483abc290 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.025215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb53b9-e126-4ad8-bf21-46f483abc290 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.025508 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb53b9-e126-4ad8-bf21-46f483abc290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.045948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-52eb53b9-e126-4ad8-bf21-46f483abc290 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:31.047216 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-52eb53b9-e126-4ad8-bf21-46f483abc290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:31.110339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1287/2576] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:31.118065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb13e-6a4a-49ed-8fdd-8032ddd2ac01 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.118355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb13e-6a4a-49ed-8fdd-8032ddd2ac01 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.118631 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cb13e-6a4a-49ed-8fdd-8032ddd2ac01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.138951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-c43cb13e-6a4a-49ed-8fdd-8032ddd2ac01 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:31.140149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c43cb13e-6a4a-49ed-8fdd-8032ddd2ac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:31.148868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1290/2577] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:31.153476 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb53b9-e126-4ad8-bf21-46f483abc290 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:31.163617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb53b9-e126-4ad8-bf21-46f483abc290 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:31.168689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:31.169331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.169636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.170304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.170969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:06:31.171269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:31.173643 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:31.190622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1288/2578] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:06:31.191710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb13e-6a4a-49ed-8fdd-8032ddd2ac01 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:31.201359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cb13e-6a4a-49ed-8fdd-8032ddd2ac01 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:31.206122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:31.206722 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.207001 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.207274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.207919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:31.209863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:31.238716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1291/2579] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:31 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:06:31.243138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:31.255874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162ea62-7e37-4ee3-abd5-4fa27515e182 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:31.324848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:31.325465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.325744 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.326016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.326661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:31.329560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:31.356637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1289/2580] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:31 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:06:31.357875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:31.368995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee150e5-6683-415f-b021-b8f47d91c756 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:31.373684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:31.374317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.374589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.374940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.375812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:31.377460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:31.391034 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1292/2581] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:31.392668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:31.402850 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f75424-1878-45c8-bf9d-ea1cc869d2ce admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:31.408438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9832ce-c94a-4df7-916f-df0cc4e4fe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:31.409089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9832ce-c94a-4df7-916f-df0cc4e4fe7d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.409451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9832ce-c94a-4df7-916f-df0cc4e4fe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.409763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9832ce-c94a-4df7-916f-df0cc4e4fe7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.410408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9832ce-c94a-4df7-916f-df0cc4e4fe7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:31.412020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9832ce-c94a-4df7-916f-df0cc4e4fe7d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:31.423964 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1290/2582] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:31.428366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:31.441998 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cab86-7de4-4bfb-a9e4-97a9e5d41290 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:31.446760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:31.447415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.447692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.447969 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] PATH_INFO: `/v3/projects/f298de5dd8fa480d8e486dc4fd7bcba5/users/6fc1536750a145e3828228f0a60758d6/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.465387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f298de5dd8fa480d8e486dc4fd7bcba5, user_id=6fc1536750a145e3828228f0a60758d6, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:31.468130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:31.484280 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1293/2583] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:31 2022] PUT /identity/v3/projects/f298de5dd8fa480d8e486dc4fd7bcba5/users/6fc1536750a145e3828228f0a60758d6/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:31.558161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.558820 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.559412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.569386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:31.616183 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] MFA Rules not processed for user `6fc1536750a145e3828228f0a60758d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:31.617703 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:31.727661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1291/2584] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:31.741382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:31.743542 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:31.824850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c7b9-b313-436e-a119-1cb4aae76951 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:31.830179 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022516f130ec4ad6a9b7c46b417a5690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:31.830852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:31.831138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:31.831413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:31.840990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:31.842491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:31.852696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1294/2585] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:35.793009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:35.794393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:35.851625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aba90-224d-4876-a347-273f899f0e81 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:35.855791 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:35.856522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:35.856826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:35.857152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:35.859697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:35.924512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:35.925989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:35.935568 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1292/2586] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:39.146960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:39.148472 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:39.164081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299cd98-fdd5-493d-8294-13ecbcee5904 service glance] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:39.168608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:39.169254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.169528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.169795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] PATH_INFO: `/v3/users/5e113c9ec968460ba070cfc1a69a25a1` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.170916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e113c9ec968460ba070cfc1a69a25a1)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:39.173628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:39.222799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e113c9ec968460ba070cfc1a69a25a1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.231535 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e113c9ec968460ba070cfc1a69a25a1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.233020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:39.249047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e113c9ec968460ba070cfc1a69a25a1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.257777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e113c9ec968460ba070cfc1a69a25a1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.258530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f54558df-3d40-47f1-b14e-99e234b7c651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e113c9ec968460ba070cfc1a69a25a1'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.264411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1295/2587] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:39 2022] DELETE /identity/v3/users/5e113c9ec968460ba070cfc1a69a25a1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:39.446495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:39.466380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4bb9f-a267-45f3-93a1-10a170a7dd1c service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:39.512128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:39.528178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:39.528848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.529132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.529406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] PATH_INFO: `/v3/projects/d7ee0d51f58e4abd8867c54458f504fc` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.530564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ee0d51f58e4abd8867c54458f504fc)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:39.533154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:39.580087 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ee0d51f58e4abd8867c54458f504fc', 'request_id': 'req-f8551fb3-1fce-45ba-822f-187b1d8127a1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.589743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ee0d51f58e4abd8867c54458f504fc', 'request_id': 'req-f8551fb3-1fce-45ba-822f-187b1d8127a1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.598530 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ee0d51f58e4abd8867c54458f504fc', 'request_id': 'req-f8551fb3-1fce-45ba-822f-187b1d8127a1'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.610038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] The token cache is being invalidate because project d7ee0d51f58e4abd8867c54458f504fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:39.610318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.610971 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f8551fb3-1fce-45ba-822f-187b1d8127a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:39.612453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1293/2588] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:39 2022] DELETE /identity/v3/projects/d7ee0d51f58e4abd8867c54458f504fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:39.617655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75c80e-4c29-4989-928d-be5ab477d81c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.617945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75c80e-4c29-4989-928d-be5ab477d81c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.618223 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75c80e-4c29-4989-928d-be5ab477d81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.637608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6c75c80e-4c29-4989-928d-be5ab477d81c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:39.638812 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6c75c80e-4c29-4989-928d-be5ab477d81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:39.696702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1296/2589] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:39.705175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650000-4917-40d1-bce2-3fbda65caf83 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.705536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650000-4917-40d1-bce2-3fbda65caf83 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.705819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650000-4917-40d1-bce2-3fbda65caf83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.727641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-27650000-4917-40d1-bce2-3fbda65caf83 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:39.729189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-27650000-4917-40d1-bce2-3fbda65caf83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:39.741153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1294/2590] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:39.745236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75c80e-4c29-4989-928d-be5ab477d81c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:39.754985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75c80e-4c29-4989-928d-be5ab477d81c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:39.759982 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:39.760584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.760888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.761158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.761796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:06:39.762094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:39.763701 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:39.779983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1297/2591] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:06:39.784890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27650000-4917-40d1-bce2-3fbda65caf83 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:39.794132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27650000-4917-40d1-bce2-3fbda65caf83 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:39.799056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:39.799658 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.799936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.800207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.800887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:39.802818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:39.831010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1295/2592] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:39 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:39.835093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:39.845165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795520d-48b1-4cbe-a845-83b0ef33bc73 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:39.905898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:39.906545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.906826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.907099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.907747 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:39.909752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:39.931286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1298/2593] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:39 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:39.935661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:39.946567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1ae40-02b8-41cd-b0b4-9e313002f25a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:39.951249 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:39.951835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.952116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.952397 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.953666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:39.955271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:39.965380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1296/2594] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:39.969198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:39.979039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58195ba7-cdce-4f55-91c2-f9eb915e02f7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:39.983651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae68bf-a16d-4008-b65f-6d6713e9f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:39.984364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae68bf-a16d-4008-b65f-6d6713e9f7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:39.984611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae68bf-a16d-4008-b65f-6d6713e9f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:39.984918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae68bf-a16d-4008-b65f-6d6713e9f7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:39.985556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae68bf-a16d-4008-b65f-6d6713e9f7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:39.987198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae68bf-a16d-4008-b65f-6d6713e9f7c1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:39.997642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1299/2595] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:40.002225 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.012129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd552be-2f4b-478f-9aa5-d37f55bc001d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.017740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.018401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.018682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.018956 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] PATH_INFO: `/v3/projects/e8b5ebc970934b919b528137221b5f46/users/01102ff0faac41e8b04e731eeeef9884/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.033480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b5ebc970934b919b528137221b5f46, user_id=01102ff0faac41e8b04e731eeeef9884, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.036565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.054844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1297/2596] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:40 2022] PUT /identity/v3/projects/e8b5ebc970934b919b528137221b5f46/users/01102ff0faac41e8b04e731eeeef9884/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:40.065765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a5fc6-e0ff-4118-a929-272103c78768 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.066043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a5fc6-e0ff-4118-a929-272103c78768 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.066311 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a5fc6-e0ff-4118-a929-272103c78768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.094059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-706a5fc6-e0ff-4118-a929-272103c78768 None None] MFA Rules not processed for user `01102ff0faac41e8b04e731eeeef9884`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:40.095150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-706a5fc6-e0ff-4118-a929-272103c78768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:40.120332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:40.161906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1300/2597] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:40.166347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.236925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1eb0f-7256-4f22-9e4c-2d83be206f3e admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.241982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.242625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.242919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.243189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.243822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.245805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.260816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:40.275074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1298/2598] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:40 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:40.279313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a5fc6-e0ff-4118-a929-272103c78768 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.290855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a5fc6-e0ff-4118-a929-272103c78768 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.352693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.353259 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.353536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.353803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.354424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.356334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.377961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1301/2599] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:40 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:40.381900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.393001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42e341-684c-4f6e-a0fb-967aadf09c7f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.397586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.398177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.398452 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.398723 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.399376 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.401029 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.412258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1299/2600] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:40.416403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.426615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b4bcc-dce0-4f49-9907-926ccafcafac admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.431327 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.431978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.432257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.432528 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] PATH_INFO: `/v3/projects/0ec6738f45694500aabd055f59a0472b/users/27e322af272e42fa88b7f125d2b8b187/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.449497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec6738f45694500aabd055f59a0472b, user_id=27e322af272e42fa88b7f125d2b8b187, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.452170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.466732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1302/2601] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:40 2022] PUT /identity/v3/projects/0ec6738f45694500aabd055f59a0472b/users/27e322af272e42fa88b7f125d2b8b187/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:40.471714 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.481750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0430f4-d6e2-42d0-9803-8d2ec957f79a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.539277 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98017-0f34-4b52-9e9a-94ff2037ccaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.539897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98017-0f34-4b52-9e9a-94ff2037ccaf admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.540144 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98017-0f34-4b52-9e9a-94ff2037ccaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.540408 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98017-0f34-4b52-9e9a-94ff2037ccaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.541095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98017-0f34-4b52-9e9a-94ff2037ccaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.542698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98017-0f34-4b52-9e9a-94ff2037ccaf admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.553494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1300/2602] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:40.557850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.569114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c10ad-0d82-47f9-85b0-93ec3f32dfa4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.573735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.574398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.574668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.574933 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] PATH_INFO: `/v3/projects/0ec6738f45694500aabd055f59a0472b/users/27e322af272e42fa88b7f125d2b8b187/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.577218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec6738f45694500aabd055f59a0472b, user_id=27e322af272e42fa88b7f125d2b8b187, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.580951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.601186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1303/2603] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:40 2022] PUT /identity/v3/projects/0ec6738f45694500aabd055f59a0472b/users/27e322af272e42fa88b7f125d2b8b187/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:06:40.617394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917600c0-f32c-4a33-9226-50de5ac5e43f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.617669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917600c0-f32c-4a33-9226-50de5ac5e43f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.618036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917600c0-f32c-4a33-9226-50de5ac5e43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.646665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-917600c0-f32c-4a33-9226-50de5ac5e43f None None] MFA Rules not processed for user `27e322af272e42fa88b7f125d2b8b187`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:40.647875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-917600c0-f32c-4a33-9226-50de5ac5e43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:40.719177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1301/2604] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:40.730841 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.732579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:40.791651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a2ec9-1578-4501-a5b4-9b2ad37e76b5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.795354 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.795956 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.796970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.797247 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.806789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.808257 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.819990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1304/2605] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:40.902330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917600c0-f32c-4a33-9226-50de5ac5e43f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:40.912028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917600c0-f32c-4a33-9226-50de5ac5e43f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:40.915622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7b99e-94e3-4377-b999-e283ade8e247 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:40.916210 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b99e-94e3-4377-b999-e283ade8e247 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:40.916482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b99e-94e3-4377-b999-e283ade8e247 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:40.916775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b99e-94e3-4377-b999-e283ade8e247 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:40.981041 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7b99e-94e3-4377-b999-e283ade8e247 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:40.982905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7b99e-94e3-4377-b999-e283ade8e247 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:40.993836 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1302/2606] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:50.653299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:50.654680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:50.680563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:50.719406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09129fb-78e9-4406-aa7a-914c075818c3 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:50.723880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:50.724491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:50.724798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:50.725065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] PATH_INFO: `/v3/users/6fc1536750a145e3828228f0a60758d6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:50.726166 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fc1536750a145e3828228f0a60758d6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:50.728059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:50.773296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fc1536750a145e3828228f0a60758d6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:50.780469 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fc1536750a145e3828228f0a60758d6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:50.792263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fc1536750a145e3828228f0a60758d6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:50.800023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fc1536750a145e3828228f0a60758d6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:50.800618 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fc1536750a145e3828228f0a60758d6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:50.806423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1305/2607] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:50 2022] DELETE /identity/v3/users/6fc1536750a145e3828228f0a60758d6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:50.810999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:50.811490 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:50.811588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:50.820223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:50.832254 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:50.833593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:50.892520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1303/2608] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:50.902234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:50.903644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:50.963008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4c7a6-1f18-4498-ade3-b2d78071bbde admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:50.966718 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3a898-71d9-48fd-8754-a3af09daeba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:50.967386 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3a898-71d9-48fd-8754-a3af09daeba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:50.967721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3a898-71d9-48fd-8754-a3af09daeba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:50.967987 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3a898-71d9-48fd-8754-a3af09daeba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:50.979368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3a898-71d9-48fd-8754-a3af09daeba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:50.980830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3a898-71d9-48fd-8754-a3af09daeba3 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:50.991110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1306/2609] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:51.269921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.271605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:51.285709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a25c1-4c87-4a53-b272-8df47bdd2bf0 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.290396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.291062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.291336 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.291606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] PATH_INFO: `/v3/projects/f298de5dd8fa480d8e486dc4fd7bcba5` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.292805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f298de5dd8fa480d8e486dc4fd7bcba5)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.294765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:51.350123 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f298de5dd8fa480d8e486dc4fd7bcba5', 'request_id': 'req-75f3649f-9687-4b11-8ad7-b869c8003542'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:51.356661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f298de5dd8fa480d8e486dc4fd7bcba5', 'request_id': 'req-75f3649f-9687-4b11-8ad7-b869c8003542'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:51.362732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f298de5dd8fa480d8e486dc4fd7bcba5', 'request_id': 'req-75f3649f-9687-4b11-8ad7-b869c8003542'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:51.373464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] The token cache is being invalidate because project f298de5dd8fa480d8e486dc4fd7bcba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:06:51.373748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:51.374365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-75f3649f-9687-4b11-8ad7-b869c8003542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:51.377146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1304/2610] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:51 2022] DELETE /identity/v3/projects/f298de5dd8fa480d8e486dc4fd7bcba5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:51.382022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7b85-3178-4883-b52a-c20e16eaa592 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.382308 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7b85-3178-4883-b52a-c20e16eaa592 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.382580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7b85-3178-4883-b52a-c20e16eaa592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.402829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-174a7b85-3178-4883-b52a-c20e16eaa592 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:51.404026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-174a7b85-3178-4883-b52a-c20e16eaa592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:51.446875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:51.466640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1307/2611] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:51.476019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fa540-2e5b-4482-86e2-89eb5c38ef69 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.476302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fa540-2e5b-4482-86e2-89eb5c38ef69 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.476592 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3fa540-2e5b-4482-86e2-89eb5c38ef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.496991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cc3fa540-2e5b-4482-86e2-89eb5c38ef69 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:51.498190 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cc3fa540-2e5b-4482-86e2-89eb5c38ef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:51.507253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1305/2612] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:51.511320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7b85-3178-4883-b52a-c20e16eaa592 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.521475 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7b85-3178-4883-b52a-c20e16eaa592 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.526035 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.526654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.526928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.527197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.527836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:06:51.528134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.529754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:51.541492 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1308/2613] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 16:06:51.545719 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fa540-2e5b-4482-86e2-89eb5c38ef69 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.555986 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3fa540-2e5b-4482-86e2-89eb5c38ef69 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.560604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.561236 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.561511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.561783 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.563571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.565519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:51.592473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1306/2614] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:51 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:51.597210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.607551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b408f-c3de-4321-89db-2a054b918cf2 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.662211 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.662856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.663063 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.663312 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.663937 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.665936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:51.688898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1309/2615] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:51 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:06:51.692900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.703112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5df17-a7c3-4526-bc37-2a142e7178ae admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.707617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.708200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.708477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.708774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.709418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.711284 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:51.721073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1307/2616] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:51.724949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.735751 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05e99-17d5-498a-968b-03fbb39c4643 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.740451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e9413-cc0b-4428-a774-016fc75c0126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.741101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e9413-cc0b-4428-a774-016fc75c0126 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.741375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e9413-cc0b-4428-a774-016fc75c0126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.741666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e9413-cc0b-4428-a774-016fc75c0126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.742299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e9413-cc0b-4428-a774-016fc75c0126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.743875 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e9413-cc0b-4428-a774-016fc75c0126 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:51.754883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1310/2617] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:06:51.758829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.769229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc526a-221e-49a3-955b-ad7dacb35513 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.773823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.774463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.774739 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.775010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] PATH_INFO: `/v3/projects/820135f4c4df447aaf1044b8a54ecffb/users/315695c8534045a9a5d8b1f8d0ff1555/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.789936 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=820135f4c4df447aaf1044b8a54ecffb, user_id=315695c8534045a9a5d8b1f8d0ff1555, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.792646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:51.806390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1308/2618] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:51 2022] PUT /identity/v3/projects/820135f4c4df447aaf1044b8a54ecffb/users/315695c8534045a9a5d8b1f8d0ff1555/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:51.819107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ba85e-566a-4dd1-af7c-4c24ebada838 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.819449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ba85e-566a-4dd1-af7c-4c24ebada838 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.819833 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ba85e-566a-4dd1-af7c-4c24ebada838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.849197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e86ba85e-566a-4dd1-af7c-4c24ebada838 None None] MFA Rules not processed for user `315695c8534045a9a5d8b1f8d0ff1555`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:51.850333 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e86ba85e-566a-4dd1-af7c-4c24ebada838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:51.913326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1311/2619] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:51.917968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:51.946745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:51.990832 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7006a-654b-4e4c-8eea-6e2f8a427475 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:51.995370 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:51.995968 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:51.996245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:51.996510 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:51.997173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:51.999075 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.025970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1309/2620] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:51 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:52.030133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ba85e-566a-4dd1-af7c-4c24ebada838 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.041367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ba85e-566a-4dd1-af7c-4c24ebada838 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.079414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:52.098382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.098991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.099320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.099539 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.100174 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.102117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.122643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1312/2621] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:52 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:52.127128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.137367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d56d6-c78d-494b-a8a1-6d34ea2e5b76 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.142711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.143302 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.143579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.143852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.144493 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.146149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.156706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1310/2622] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:52.160645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.170797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3feac6-54bf-41e6-870a-425743ff6472 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.176345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8c04c-ca6f-4a2f-aaf5-c9ec9732ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.177066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8c04c-ca6f-4a2f-aaf5-c9ec9732ce0f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.177429 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8c04c-ca6f-4a2f-aaf5-c9ec9732ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.177697 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8c04c-ca6f-4a2f-aaf5-c9ec9732ce0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.178406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8c04c-ca6f-4a2f-aaf5-c9ec9732ce0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.180138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8c04c-ca6f-4a2f-aaf5-c9ec9732ce0f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.191299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1313/2623] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:06:52.195391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.205498 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab56c7-bc83-406e-a559-5e70648e0146 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.210150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.211063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.211339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.211613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] PATH_INFO: `/v3/projects/a93f116c4f47414d9cad55d480131123/users/79afa01e5108425fb7fa76e7bbdcf368/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.227752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93f116c4f47414d9cad55d480131123, user_id=79afa01e5108425fb7fa76e7bbdcf368, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.231806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.244637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1311/2624] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:52 2022] PUT /identity/v3/projects/a93f116c4f47414d9cad55d480131123/users/79afa01e5108425fb7fa76e7bbdcf368/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:52.258628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc0ebe-eef0-4afd-87f0-3314caf3c1d7 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.258628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc0ebe-eef0-4afd-87f0-3314caf3c1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.258628 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc0ebe-eef0-4afd-87f0-3314caf3c1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.288669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-91dc0ebe-eef0-4afd-87f0-3314caf3c1d7 None None] MFA Rules not processed for user `79afa01e5108425fb7fa76e7bbdcf368`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:52.289983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-91dc0ebe-eef0-4afd-87f0-3314caf3c1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:52.365079 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1314/2625] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:52.369337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.433668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76240da-698b-4793-883b-1d6da950066a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.439326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.439930 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.440204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.440512 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.441168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.443189 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.471401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1312/2626] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:52.475476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc0ebe-eef0-4afd-87f0-3314caf3c1d7 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.487651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc0ebe-eef0-4afd-87f0-3314caf3c1d7 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.560099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.560868 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.561213 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.561562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.562363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.564951 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.592185 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1315/2627] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:06:52 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:06:52.594489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.605838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9eb65-fc1b-4965-8c2d-007373dc5aaa admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.612367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.613066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.613343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.613610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.614340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.616207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.628797 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1313/2628] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:52.633307 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.644710 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3bc95e-ffbd-444f-b5bd-24354f25a6a7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.649837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.650506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.650964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.651241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] PATH_INFO: `/v3/projects/b987a7a592fe40c7affc2657fa145d6b/users/cd06916b588e4fdeb821937bef608539/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.669236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b987a7a592fe40c7affc2657fa145d6b, user_id=cd06916b588e4fdeb821937bef608539, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.673170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.688971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1316/2629] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:52 2022] PUT /identity/v3/projects/b987a7a592fe40c7affc2657fa145d6b/users/cd06916b588e4fdeb821937bef608539/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:06:52.693603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.705345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563058c-dd37-4f43-a1d6-88457969d6b5 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.759295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:52.773946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444270d-36e9-435a-abb2-07a71fa6e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.774549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444270d-36e9-435a-abb2-07a71fa6e9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.774826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444270d-36e9-435a-abb2-07a71fa6e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.775094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444270d-36e9-435a-abb2-07a71fa6e9d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.775732 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444270d-36e9-435a-abb2-07a71fa6e9d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.777406 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444270d-36e9-435a-abb2-07a71fa6e9d5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.790708 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1314/2630] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:06:52.791980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.802663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a650c-6a25-4a62-94a0-3eeade855945 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:52.807465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:52.808111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.808388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.808657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] PATH_INFO: `/v3/projects/b987a7a592fe40c7affc2657fa145d6b/users/cd06916b588e4fdeb821937bef608539/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.810627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b987a7a592fe40c7affc2657fa145d6b, user_id=cd06916b588e4fdeb821937bef608539, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:52.813863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:52.821376 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:52.826839 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1317/2631] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:06:52 2022] PUT /identity/v3/projects/b987a7a592fe40c7affc2657fa145d6b/users/cd06916b588e4fdeb821937bef608539/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:52.838678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1000d2-a6ea-4fe4-b3bc-64d48cacb295 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:52.838960 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1000d2-a6ea-4fe4-b3bc-64d48cacb295 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:52.839312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1000d2-a6ea-4fe4-b3bc-64d48cacb295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:52.867679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1a1000d2-a6ea-4fe4-b3bc-64d48cacb295 None None] MFA Rules not processed for user `cd06916b588e4fdeb821937bef608539`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:52.869052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1a1000d2-a6ea-4fe4-b3bc-64d48cacb295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:52.936009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1315/2632] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:52.948322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:52.950143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:53.007966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d25fc9-2600-4d48-88f2-ea52edd2c88f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:53.011663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:53.012262 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:53.012538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:53.012845 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:53.070533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:53.071971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:53.081070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1318/2633] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:54.582632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1000d2-a6ea-4fe4-b3bc-64d48cacb295 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:54.592611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1000d2-a6ea-4fe4-b3bc-64d48cacb295 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:54.596906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:54.597503 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:54.597781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:54.598054 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:54.661872 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:54.663327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:54.672380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1316/2634] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:55.281746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:55.290532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284b987-7284-40ae-8d91-0452d0d7ac3f service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:55.294206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d9be5-9248-4a71-818b-69c634ed2eaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:55.294733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9be5-9248-4a71-818b-69c634ed2eaa service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:55.295003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9be5-9248-4a71-818b-69c634ed2eaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:55.295264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d9be5-9248-4a71-818b-69c634ed2eaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:55.305121 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d9be5-9248-4a71-818b-69c634ed2eaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:55.306534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d9be5-9248-4a71-818b-69c634ed2eaa service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:55.316952 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1319/2635] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:59.300399 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:59.367091 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1228-ebc0-4d8e-b8b8-ca9063be88ce service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:59.371603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:59.372251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:59.372533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:59.373039 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] PATH_INFO: `/v3/users/315695c8534045a9a5d8b1f8d0ff1555` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:59.374076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=315695c8534045a9a5d8b1f8d0ff1555)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:59.376106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:59.386561 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:59.422939 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '315695c8534045a9a5d8b1f8d0ff1555'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:59.423582 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '315695c8534045a9a5d8b1f8d0ff1555'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:59.432068 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '315695c8534045a9a5d8b1f8d0ff1555'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:59.444414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '315695c8534045a9a5d8b1f8d0ff1555'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:59.451081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '315695c8534045a9a5d8b1f8d0ff1555'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:59.456945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1317/2636] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:06:59 2022] DELETE /identity/v3/users/315695c8534045a9a5d8b1f8d0ff1555 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:06:59.461756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:59.462245 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:59.462382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:59.481939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:06:59.483057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:59.541361 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1320/2637] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:06:59.551675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:59.553295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:59.611499 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97794529-47a9-41f5-9b45-88d70ba2ee0b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:59.615232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:59.615830 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:59.616105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:59.616371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:59.625412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:59.626847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:59.636177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1318/2638] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:06:59.904547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:06:59.906633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:06:59.918242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17582d92-f556-4e64-b98f-c7dda917e402 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:06:59.923132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:06:59.923753 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:06:59.924026 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:06:59.924961 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] PATH_INFO: `/v3/projects/820135f4c4df447aaf1044b8a54ecffb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:06:59.926266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] RBAC: Authorizing `identity:delete_project(project_id=820135f4c4df447aaf1044b8a54ecffb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:06:59.928168 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:06:59.970895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:06:59.980548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820135f4c4df447aaf1044b8a54ecffb', 'request_id': 'req-1e24c370-0453-4d59-ab67-3d234fb67add'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:59.988255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820135f4c4df447aaf1044b8a54ecffb', 'request_id': 'req-1e24c370-0453-4d59-ab67-3d234fb67add'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:06:59.994279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820135f4c4df447aaf1044b8a54ecffb', 'request_id': 'req-1e24c370-0453-4d59-ab67-3d234fb67add'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.006021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] The token cache is being invalidate because project 820135f4c4df447aaf1044b8a54ecffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:00.006305 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.006919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.008419 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1321/2639] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:06:59 2022] DELETE /identity/v3/projects/820135f4c4df447aaf1044b8a54ecffb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:00.012366 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:00.023046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b455a-11da-48f2-a9b8-4b1b90ae4307 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:00.088390 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:00.089078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:00.089353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:00.089623 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] PATH_INFO: `/v3/users/79afa01e5108425fb7fa76e7bbdcf368` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:00.090764 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79afa01e5108425fb7fa76e7bbdcf368)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:00.092692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:00.130826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:00.146013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79afa01e5108425fb7fa76e7bbdcf368'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.146667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79afa01e5108425fb7fa76e7bbdcf368'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.153752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79afa01e5108425fb7fa76e7bbdcf368'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.166295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79afa01e5108425fb7fa76e7bbdcf368'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.173209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79afa01e5108425fb7fa76e7bbdcf368'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.179081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1319/2640] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:00 2022] DELETE /identity/v3/users/79afa01e5108425fb7fa76e7bbdcf368 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:00.451326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:00.453197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:00.466873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24c370-0453-4d59-ab67-3d234fb67add admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:00.523971 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:00.524524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:00.524824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:00.525098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] PATH_INFO: `/v3/projects/a93f116c4f47414d9cad55d480131123` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:00.526207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93f116c4f47414d9cad55d480131123)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:00.528106 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:00.576182 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93f116c4f47414d9cad55d480131123', 'request_id': 'req-8ce10add-f28c-456b-8e51-3e9c0e93d738'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.585959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93f116c4f47414d9cad55d480131123', 'request_id': 'req-8ce10add-f28c-456b-8e51-3e9c0e93d738'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.591934 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93f116c4f47414d9cad55d480131123', 'request_id': 'req-8ce10add-f28c-456b-8e51-3e9c0e93d738'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.617656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] The token cache is being invalidate because project a93f116c4f47414d9cad55d480131123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:00.617945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.618715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.620498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1322/2641] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:00 2022] DELETE /identity/v3/projects/a93f116c4f47414d9cad55d480131123 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:00.625155 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:00.626662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:00.696923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca942c-4b1f-4b1f-9efd-1d6ba3979ac7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:00.759963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:00.760603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:00.760923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:00.761201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] PATH_INFO: `/v3/users/cd06916b588e4fdeb821937bef608539` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:00.762355 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd06916b588e4fdeb821937bef608539)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:00.764292 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:00.814013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd06916b588e4fdeb821937bef608539'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.814668 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd06916b588e4fdeb821937bef608539'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.822274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd06916b588e4fdeb821937bef608539'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.828233 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:00.838270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd06916b588e4fdeb821937bef608539'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.846468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a1513b5c-67c0-48a2-8b38-ef81c1db7e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd06916b588e4fdeb821937bef608539'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:00.853083 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1320/2642] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:00 2022] DELETE /identity/v3/users/cd06916b588e4fdeb821937bef608539 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:01.154573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.156104 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:01.217667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:01.220421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce10add-f28c-456b-8e51-3e9c0e93d738 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:01.284830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:01.284830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.284830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.284830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] PATH_INFO: `/v3/projects/b987a7a592fe40c7affc2657fa145d6b` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.285252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b987a7a592fe40c7affc2657fa145d6b)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:01.285821 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:01.334788 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b987a7a592fe40c7affc2657fa145d6b', 'request_id': 'req-784fded2-afa1-4d2f-a8f7-3619b5fbb209'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:01.342431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b987a7a592fe40c7affc2657fa145d6b', 'request_id': 'req-784fded2-afa1-4d2f-a8f7-3619b5fbb209'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:01.348365 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b987a7a592fe40c7affc2657fa145d6b', 'request_id': 'req-784fded2-afa1-4d2f-a8f7-3619b5fbb209'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:01.359756 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] The token cache is being invalidate because project b987a7a592fe40c7affc2657fa145d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:01.360038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:01.360672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-784fded2-afa1-4d2f-a8f7-3619b5fbb209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:01.362258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1323/2643] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:01 2022] DELETE /identity/v3/projects/b987a7a592fe40c7affc2657fa145d6b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:01.367246 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d957039-3c48-4ae3-9366-b53cd0832870 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.367535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d957039-3c48-4ae3-9366-b53cd0832870 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.367809 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d957039-3c48-4ae3-9366-b53cd0832870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.424747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5d957039-3c48-4ae3-9366-b53cd0832870 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:01.425943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5d957039-3c48-4ae3-9366-b53cd0832870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:01.490823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1321/2644] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:01.500568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338617-2844-4501-824a-3943f989d951 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.501084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338617-2844-4501-824a-3943f989d951 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.501178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338617-2844-4501-824a-3943f989d951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.520569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-21338617-2844-4501-824a-3943f989d951 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:01.521804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-21338617-2844-4501-824a-3943f989d951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:01.529679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1324/2645] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:01.534429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d957039-3c48-4ae3-9366-b53cd0832870 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.543962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d957039-3c48-4ae3-9366-b53cd0832870 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:01.549220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:01.549807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.550080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.550348 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.550984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:07:01.551281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:01.552935 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:01.564242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1322/2646] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:07:01.568414 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21338617-2844-4501-824a-3943f989d951 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.577613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21338617-2844-4501-824a-3943f989d951 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:01.582328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:01.582909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.583183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.583446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.584071 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:01.586374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:01.614322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1325/2647] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:01 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:07:01.618411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.628351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d723372-933f-49bd-b2e2-55cb84bb863c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:01.685021 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:01.685622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.685895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.686161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.686799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:01.688863 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:01.711073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1323/2648] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:01 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:01.714240 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.725073 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0806b8-8ff8-450a-8a44-6ca5c619d24e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:01.729594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:01.730178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.730452 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.730717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.731351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:01.732968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:01.743236 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1326/2649] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:01.746972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.756913 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de09e0-dd23-4d05-b87c-1713502ff984 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:01.761774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44117499-d660-44a3-9b09-1a7a20c8cad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:01.762357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117499-d660-44a3-9b09-1a7a20c8cad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.762638 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117499-d660-44a3-9b09-1a7a20c8cad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.762905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44117499-d660-44a3-9b09-1a7a20c8cad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.763539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44117499-d660-44a3-9b09-1a7a20c8cad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:01.765305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44117499-d660-44a3-9b09-1a7a20c8cad0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:01.775062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1324/2650] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:01.778991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.789202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab23b06-2c50-4df9-8590-3c7929abc670 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:01.793733 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:01.794360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.794634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.794901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] PATH_INFO: `/v3/projects/a7ffe330a5d2482e961c93077d13cc5b/users/7fe42f8d88c540418580fb8c668bf2de/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.810808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ffe330a5d2482e961c93077d13cc5b, user_id=7fe42f8d88c540418580fb8c668bf2de, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:01.814045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:01.826755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1327/2651] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:01 2022] PUT /identity/v3/projects/a7ffe330a5d2482e961c93077d13cc5b/users/7fe42f8d88c540418580fb8c668bf2de/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:01.832609 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:01.832876 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:01.833169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:01.852595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:01.853831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:01.862666 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:01.915769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1325/2652] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:01.926796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:01.928484 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:01.960374 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:02.003926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993438b2-8d80-4763-b586-7410a8e07c6f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:02.007642 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c62411-7fe6-4f05-a842-919e0c59eac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:02.008238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62411-7fe6-4f05-a842-919e0c59eac6 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:02.008514 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62411-7fe6-4f05-a842-919e0c59eac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:02.008820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c62411-7fe6-4f05-a842-919e0c59eac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:02.017992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c62411-7fe6-4f05-a842-919e0c59eac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:02.019428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c62411-7fe6-4f05-a842-919e0c59eac6 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:02.028792 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1328/2653] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:07:04.892550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:04.894197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:05.007467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e3e17-0817-43f5-910b-78c5ad171875 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:05.012047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:05.012818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:05.013141 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:05.013484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:05.015496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:05.118118 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:05.119580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:05.130494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1326/2654] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:06.525120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd260f4-8ca0-45d3-9359-bf806def6f51 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:06.525569 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd260f4-8ca0-45d3-9359-bf806def6f51 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:06.525655 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd260f4-8ca0-45d3-9359-bf806def6f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:06.593781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-3cd260f4-8ca0-45d3-9359-bf806def6f51 None None] MFA Rules not processed for user `7fe42f8d88c540418580fb8c668bf2de`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:06.595002 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3cd260f4-8ca0-45d3-9359-bf806def6f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:06.678160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1329/2655] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:06.682422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:06.696018 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d6941-2f5f-479e-8990-5defb59026b8 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:06.700611 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:06.701276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:06.701552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:06.701822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:06.702492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:06.704416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:06.732436 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1327/2656] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:06 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:06.737313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd260f4-8ca0-45d3-9359-bf806def6f51 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:06.748682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd260f4-8ca0-45d3-9359-bf806def6f51 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:06.819124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:06.819850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:06.820124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:06.820390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:06.821124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:06.823050 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:06.876109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1330/2657] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:06 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:06.877941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:06.889150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40740b45-b9e5-4e0d-860c-6bf4af6f1a96 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:06.895929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:06.900289 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:06.900445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:06.900702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:06.901386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:06.903430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:06.940145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1328/2658] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:07:06.940996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:06.951495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5220a-1fbf-4d2a-b585-8b6762f91dd4 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:06.956811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:06.957479 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:06.957752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:06.958031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] PATH_INFO: `/v3/projects/4fc33625256646a3b8bf4158052e669a/users/8583f7200658447b8be5a40654042622/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:06.979424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc33625256646a3b8bf4158052e669a, user_id=8583f7200658447b8be5a40654042622, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:06.982397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:07.001132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1331/2659] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:06 2022] PUT /identity/v3/projects/4fc33625256646a3b8bf4158052e669a/users/8583f7200658447b8be5a40654042622/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:07.005415 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:07.016145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d5726-356b-48b6-9e16-5416839f5f54 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:07.075066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:07.107457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:07.108073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:07.108335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:07.109297 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:07.109959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:07.111736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:07.129019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1329/2660] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:07.133456 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:07.144978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0360362-1f12-4e87-baaf-7bcefbbb1963 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:07.150039 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df5108-6a9c-47f6-87a8-df93f3c5654d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:07.150860 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df5108-6a9c-47f6-87a8-df93f3c5654d admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:07.151172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df5108-6a9c-47f6-87a8-df93f3c5654d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:07.151453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df5108-6a9c-47f6-87a8-df93f3c5654d admin admin] PATH_INFO: `/v3/projects/4fc33625256646a3b8bf4158052e669a/users/8583f7200658447b8be5a40654042622/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:07.153500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df5108-6a9c-47f6-87a8-df93f3c5654d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc33625256646a3b8bf4158052e669a, user_id=8583f7200658447b8be5a40654042622, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:07.157129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df5108-6a9c-47f6-87a8-df93f3c5654d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:07.163248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:07.170068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1332/2661] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:07 2022] PUT /identity/v3/projects/4fc33625256646a3b8bf4158052e669a/users/8583f7200658447b8be5a40654042622/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:08.541286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:08.543545 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:08.713700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979adc93-0ca9-4acf-8df1-5a7a2b9b78a2 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:08.926794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:08.927435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:08.932841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:08.933162 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:08.941794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:08.976924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:08.984304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:09.014590 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1330/2662] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:11.885242 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107220-636a-43ca-a436-47aa53e91bed None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:11.885803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107220-636a-43ca-a436-47aa53e91bed None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:11.885803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107220-636a-43ca-a436-47aa53e91bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:11.924054 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-18107220-636a-43ca-a436-47aa53e91bed None None] MFA Rules not processed for user `8583f7200658447b8be5a40654042622`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:11.925624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-18107220-636a-43ca-a436-47aa53e91bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:12.029062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1333/2663] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:12.049132 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:12.050953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:12.125946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:12.140104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104132e-4ffd-4352-850e-f6412c30d02d service placement] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:12.147145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '022516f130ec4ad6a9b7c46b417a5690', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:12.148070 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:12.148625 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:12.149095 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:12.233645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:12.235234 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:12.244774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1331/2664] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:12.437075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02795-3b34-4dbd-be2a-13438298cb35 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:12.437895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02795-3b34-4dbd-be2a-13438298cb35 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:12.438197 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02795-3b34-4dbd-be2a-13438298cb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:12.475983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-07f02795-3b34-4dbd-be2a-13438298cb35 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:12.479147 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-07f02795-3b34-4dbd-be2a-13438298cb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:12.559290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1334/2665] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:07:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:07:12.613552 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:12.615244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:12.699221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d4fd9-50a6-4867-96fa-cc219d9cedda service glance] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:12.705733 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb857ca-ed17-49f1-8fa7-9d4a0f06b0ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:12.706550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb857ca-ed17-49f1-8fa7-9d4a0f06b0ef service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:12.706898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb857ca-ed17-49f1-8fa7-9d4a0f06b0ef service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:12.707259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb857ca-ed17-49f1-8fa7-9d4a0f06b0ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:12.717358 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb857ca-ed17-49f1-8fa7-9d4a0f06b0ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:12.719016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb857ca-ed17-49f1-8fa7-9d4a0f06b0ef service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:12.727539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1332/2666] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:07:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:07:14.608487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3339db-a074-4e1f-88fd-bf4d7c498d6a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:14.608950 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3339db-a074-4e1f-88fd-bf4d7c498d6a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:14.609256 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3339db-a074-4e1f-88fd-bf4d7c498d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:14.629617 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-7f3339db-a074-4e1f-88fd-bf4d7c498d6a None None] MFA Rules not processed for user `7377a1bca391446eb36b2a3233a7d471`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:14.630926 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7f3339db-a074-4e1f-88fd-bf4d7c498d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:14.748779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1335/2667] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:22.724025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=60 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:24.738683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:25.471786 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:25.472322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:25.472322 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:25.493824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] MFA Rules not processed for user `7fe42f8d88c540418580fb8c668bf2de`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:25.495275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:25.503492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1333/2668] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:26.439434 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d2922-b428-4da5-8624-ec09f9be6e33 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:26.439957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d2922-b428-4da5-8624-ec09f9be6e33 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:26.440345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d2922-b428-4da5-8624-ec09f9be6e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:26.459690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-056d2922-b428-4da5-8624-ec09f9be6e33 None None] MFA Rules not processed for user `4cc1db8ef8c44763a0e5f62a3404a834`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:26.461181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-056d2922-b428-4da5-8624-ec09f9be6e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:26.534165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1336/2669] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:27.829823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:27.831501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:27.870922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669d43d-8083-4f1f-9982-b04a5653fbd4 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:27.871789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669d43d-8083-4f1f-9982-b04a5653fbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:27.872062 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669d43d-8083-4f1f-9982-b04a5653fbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:27.898260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-f669d43d-8083-4f1f-9982-b04a5653fbd4 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:27.899499 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-f669d43d-8083-4f1f-9982-b04a5653fbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:27.910096 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1337/2670] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:07:27.918560 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcf07e-4ae9-4f4a-a2a4-73340cd6afc9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:27.924458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:27.925139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:27.925417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:27.925687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] PATH_INFO: `/v3/users/01102ff0faac41e8b04e731eeeef9884` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:27.926845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01102ff0faac41e8b04e731eeeef9884)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:27.928645 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669d43d-8083-4f1f-9982-b04a5653fbd4 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:27.928921 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:27.942459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669d43d-8083-4f1f-9982-b04a5653fbd4 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:27.949421 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6793d47-cccd-4ffb-b9ef-741ee7629ab3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:27.950122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793d47-cccd-4ffb-b9ef-741ee7629ab3 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:27.950634 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793d47-cccd-4ffb-b9ef-741ee7629ab3 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:27.951201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6793d47-cccd-4ffb-b9ef-741ee7629ab3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:27.962352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6793d47-cccd-4ffb-b9ef-741ee7629ab3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:27.963989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6793d47-cccd-4ffb-b9ef-741ee7629ab3 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:27.974789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1338/2671] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:07:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 16:07:27.996788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01102ff0faac41e8b04e731eeeef9884'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:27.997823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01102ff0faac41e8b04e731eeeef9884'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:28.010112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01102ff0faac41e8b04e731eeeef9884'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:28.026367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01102ff0faac41e8b04e731eeeef9884'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:28.049699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01102ff0faac41e8b04e731eeeef9884'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:28.064110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1334/2672] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:27 2022] DELETE /identity/v3/users/01102ff0faac41e8b04e731eeeef9884 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:28.069453 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:28.069736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:28.070013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:28.112411 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:28.113870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:28.258856 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1339/2673] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:28.270750 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:28.273354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:28.440470 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec98cc7-85e0-4b77-8dc5-fa7827751e60 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:28.447069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:28.447711 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:28.448812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:28.449097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:28.470505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:28.479006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:28.481787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:28.509928 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1335/2674] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:29.031357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:29.032966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:29.047692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fff7a2-2d57-4712-a08e-946020f4eeba None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:29.056286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:29.057129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:29.057565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:29.057706 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] PATH_INFO: `/v3/projects/e8b5ebc970934b919b528137221b5f46` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:29.059688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8b5ebc970934b919b528137221b5f46)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:29.063985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:29.127041 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8b5ebc970934b919b528137221b5f46', 'request_id': 'req-7165ec90-56c6-4f31-be8d-54e7bfa88165'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.136227 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8b5ebc970934b919b528137221b5f46', 'request_id': 'req-7165ec90-56c6-4f31-be8d-54e7bfa88165'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.144309 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8b5ebc970934b919b528137221b5f46', 'request_id': 'req-7165ec90-56c6-4f31-be8d-54e7bfa88165'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.156580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] The token cache is being invalidate because project e8b5ebc970934b919b528137221b5f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:29.156925 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.157593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.159103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1340/2675] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:29 2022] DELETE /identity/v3/projects/e8b5ebc970934b919b528137221b5f46 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:29.163213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:29.173239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd9490-227a-46d5-8779-d54f9e5e03eb service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:29.234596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:29.235329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:29.235705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:29.236009 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] PATH_INFO: `/v3/users/27e322af272e42fa88b7f125d2b8b187` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:29.237271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e322af272e42fa88b7f125d2b8b187)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:29.239343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:29.295161 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e322af272e42fa88b7f125d2b8b187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.296172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e322af272e42fa88b7f125d2b8b187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.311196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e322af272e42fa88b7f125d2b8b187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.329471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e322af272e42fa88b7f125d2b8b187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.336917 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e45bbb16-0dc0-41f8-b57d-57c20b94fe11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e322af272e42fa88b7f125d2b8b187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.343152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1336/2676] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:29 2022] DELETE /identity/v3/users/27e322af272e42fa88b7f125d2b8b187 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:29.709901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:29.712060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:29.727275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165ec90-56c6-4f31-be8d-54e7bfa88165 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:29.736049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:29.821021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:29.821938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:29.822381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:29.825201 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] PATH_INFO: `/v3/projects/0ec6738f45694500aabd055f59a0472b` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:29.826913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec6738f45694500aabd055f59a0472b)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:29.830406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:29.914119 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec6738f45694500aabd055f59a0472b', 'request_id': 'req-df2134e1-a72f-40db-82f8-9de5ed08269f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.930351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec6738f45694500aabd055f59a0472b', 'request_id': 'req-df2134e1-a72f-40db-82f8-9de5ed08269f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.939122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec6738f45694500aabd055f59a0472b', 'request_id': 'req-df2134e1-a72f-40db-82f8-9de5ed08269f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.957716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] The token cache is being invalidate because project 0ec6738f45694500aabd055f59a0472b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:29.958066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.959004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-df2134e1-a72f-40db-82f8-9de5ed08269f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:29.962658 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1341/2677] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:29 2022] DELETE /identity/v3/projects/0ec6738f45694500aabd055f59a0472b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:29.969317 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c20ab-5e22-4a59-9be9-276208dbd764 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:29.969940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c20ab-5e22-4a59-9be9-276208dbd764 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:29.970409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c20ab-5e22-4a59-9be9-276208dbd764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.011074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-a05c20ab-5e22-4a59-9be9-276208dbd764 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:30.013094 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a05c20ab-5e22-4a59-9be9-276208dbd764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:30.088911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1337/2678] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:30.093743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e21cf-34d1-4164-a3b9-ddf39249e992 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.094251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e21cf-34d1-4164-a3b9-ddf39249e992 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.094359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e21cf-34d1-4164-a3b9-ddf39249e992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.117275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-aa0e21cf-34d1-4164-a3b9-ddf39249e992 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:30.118898 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-aa0e21cf-34d1-4164-a3b9-ddf39249e992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:30.128777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1342/2679] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:30.132911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c20ab-5e22-4a59-9be9-276208dbd764 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.144378 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c20ab-5e22-4a59-9be9-276208dbd764 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.162770 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.163824 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.164911 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.165628 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.166457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:07:30.167052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.169395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.183817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1338/2680] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:07:30.187789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e21cf-34d1-4164-a3b9-ddf39249e992 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.197783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e21cf-34d1-4164-a3b9-ddf39249e992 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.202861 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.203555 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.203903 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.204246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.205030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.207239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.239832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1343/2681] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:30 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:30.245522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.251360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:30.257229 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf505c6-8c61-4669-988b-5ab568de222c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.335641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.336353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.336697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.337073 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.337814 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.339849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.363817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1339/2682] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:30 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:07:30.368003 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.380830 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a48ea-7729-4f7d-8696-d68d7ecb15cd admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.385614 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.386228 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.386507 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.386774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.387420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.389798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.400516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1344/2683] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:30.404785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.415143 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37413c6-243c-4579-8ddf-8fa4c8dd3e39 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.421027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8d2d1-237a-4d80-bed4-298224ce2bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.421758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8d2d1-237a-4d80-bed4-298224ce2bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.422127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8d2d1-237a-4d80-bed4-298224ce2bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.422506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8d2d1-237a-4d80-bed4-298224ce2bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.423251 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8d2d1-237a-4d80-bed4-298224ce2bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.425063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8d2d1-237a-4d80-bed4-298224ce2bb9 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.437437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1340/2684] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:30.441584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.457636 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b867bd-a720-43cd-82e6-36e8653997f3 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.464128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.465067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.465516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.465838 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] PATH_INFO: `/v3/projects/f6905034b7424e8d86fd426d546bcd85/users/b5ebe9427e2b4c63b62772ed7fdc3175/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.483678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6905034b7424e8d86fd426d546bcd85, user_id=b5ebe9427e2b4c63b62772ed7fdc3175, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.486558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.502635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1345/2685] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:30 2022] PUT /identity/v3/projects/f6905034b7424e8d86fd426d546bcd85/users/b5ebe9427e2b4c63b62772ed7fdc3175/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:30.513275 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61600-5c65-4379-a3b5-3d18f434bbb9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.513866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61600-5c65-4379-a3b5-3d18f434bbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.513866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61600-5c65-4379-a3b5-3d18f434bbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.546127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-aab61600-5c65-4379-a3b5-3d18f434bbb9 None None] MFA Rules not processed for user `b5ebe9427e2b4c63b62772ed7fdc3175`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:30.547343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-aab61600-5c65-4379-a3b5-3d18f434bbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:30.650555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1341/2686] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:30.651442 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.714767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:30.745546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d862771-d75d-4fc0-9c0e-c60ee8d76af5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.753559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.754565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.755009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.755382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.756424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.759044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.801796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1346/2687] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:30 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:30.805869 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab61600-5c65-4379-a3b5-3d18f434bbb9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.818269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab61600-5c65-4379-a3b5-3d18f434bbb9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.883450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.884062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.884335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.884602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.885279 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.887208 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.906116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:30.912230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1342/2688] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:30 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:30.915872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.927823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e3efa-460f-45fb-85e9-e4c2aac600c5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.934202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.934964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.935238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.936059 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.937081 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.939004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:30.953389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1347/2689] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:30.957619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:30.969453 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd3657-c86c-45e1-ba65-8770c3618753 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:30.974485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:30.975291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:30.975585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:30.975920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] PATH_INFO: `/v3/projects/a7e8d4314e034287b3a6816d21847926/users/acb824b0a11f414fb42c534db715ef4d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:30.995457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e8d4314e034287b3a6816d21847926, user_id=acb824b0a11f414fb42c534db715ef4d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:30.998496 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:31.023180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1343/2690] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:30 2022] PUT /identity/v3/projects/a7e8d4314e034287b3a6816d21847926/users/acb824b0a11f414fb42c534db715ef4d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:31.024266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:31.036425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564cf72-b8bc-4a4d-aef9-c3b65243ee59 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:31.129366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e34dd-c9be-4e52-8c1e-3fa8cd93f4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:31.130654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e34dd-c9be-4e52-8c1e-3fa8cd93f4be admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:31.131012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e34dd-c9be-4e52-8c1e-3fa8cd93f4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:31.131350 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e34dd-c9be-4e52-8c1e-3fa8cd93f4be admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:31.132080 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e34dd-c9be-4e52-8c1e-3fa8cd93f4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:31.134016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e34dd-c9be-4e52-8c1e-3fa8cd93f4be admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:31.151349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1348/2691] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:31.156467 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:31.167442 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31842ae9-04d4-4894-90f0-eb8353032469 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:31.172976 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:31.173962 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:31.174347 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:31.174718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] PATH_INFO: `/v3/projects/a7e8d4314e034287b3a6816d21847926/users/acb824b0a11f414fb42c534db715ef4d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:31.176885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e8d4314e034287b3a6816d21847926, user_id=acb824b0a11f414fb42c534db715ef4d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:31.179718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:31.213276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1344/2692] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:31 2022] PUT /identity/v3/projects/a7e8d4314e034287b3a6816d21847926/users/acb824b0a11f414fb42c534db715ef4d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:31.241093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6782b4-4ef9-4c1c-b17c-15bd69253dd8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:31.241680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6782b4-4ef9-4c1c-b17c-15bd69253dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:31.242183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6782b4-4ef9-4c1c-b17c-15bd69253dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:31.276786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-8b6782b4-4ef9-4c1c-b17c-15bd69253dd8 None None] MFA Rules not processed for user `acb824b0a11f414fb42c534db715ef4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:31.278065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8b6782b4-4ef9-4c1c-b17c-15bd69253dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:31.396755 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1349/2693] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:31.409970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:31.411712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:31.507839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f1aab-42e4-45ee-8ab8-efd72b4fb380 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:31.511533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f44f5a-f4a1-43cd-8756-72acfc2c61ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:31.512135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f44f5a-f4a1-43cd-8756-72acfc2c61ec service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:31.512417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f44f5a-f4a1-43cd-8756-72acfc2c61ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:31.512686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f44f5a-f4a1-43cd-8756-72acfc2c61ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:31.523062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f44f5a-f4a1-43cd-8756-72acfc2c61ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:31.524564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f44f5a-f4a1-43cd-8756-72acfc2c61ec service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:31.535056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1345/2694] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:31.565893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6782b4-4ef9-4c1c-b17c-15bd69253dd8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:31.607678 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:31.645622 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6782b4-4ef9-4c1c-b17c-15bd69253dd8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:31.653716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:31.654393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:31.654666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:31.654936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] PATH_INFO: `/v3/users/b5ebe9427e2b4c63b62772ed7fdc3175` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:31.656160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ebe9427e2b4c63b62772ed7fdc3175)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:31.658144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:31.717885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ebe9427e2b4c63b62772ed7fdc3175'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:31.727412 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ebe9427e2b4c63b62772ed7fdc3175'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:31.744936 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ebe9427e2b4c63b62772ed7fdc3175'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:31.752075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ebe9427e2b4c63b62772ed7fdc3175'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:31.752886 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ebe9427e2b4c63b62772ed7fdc3175'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:31.758829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1350/2695] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:31 2022] DELETE /identity/v3/users/b5ebe9427e2b4c63b62772ed7fdc3175 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:31.763441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:31.764596 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:31.764995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:31.787330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:31.788704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:31.903464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1346/2696] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:07:31.915285 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:31.917190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:32.011078 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341f20a-7f63-484b-8415-8d9b330d2fde admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:32.018214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:32.018777 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:32.018904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:32.021064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:32.031075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:32.032899 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:32.046502 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1351/2697] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:32.407154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:32.408653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:32.428169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286de03-7712-4945-a0b7-e9877f53f32e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:32.439946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:32.440607 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:32.441078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:32.441213 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] PATH_INFO: `/v3/projects/f6905034b7424e8d86fd426d546bcd85` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:32.443312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6905034b7424e8d86fd426d546bcd85)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:32.446927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:32.565517 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:32.578583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6905034b7424e8d86fd426d546bcd85', 'request_id': 'req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.595164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6905034b7424e8d86fd426d546bcd85', 'request_id': 'req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.605963 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6905034b7424e8d86fd426d546bcd85', 'request_id': 'req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.623774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] The token cache is being invalidate because project f6905034b7424e8d86fd426d546bcd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:32.624072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.624924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.626696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1347/2698] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:32 2022] DELETE /identity/v3/projects/f6905034b7424e8d86fd426d546bcd85 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:32.631317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:32.644888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4679d98-2e65-4710-8723-1e49f227fec4 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:32.711848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:32.722473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:32.723326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:32.723790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:32.724103 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] PATH_INFO: `/v3/users/acb824b0a11f414fb42c534db715ef4d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:32.725430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb824b0a11f414fb42c534db715ef4d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:32.727398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:32.776086 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb824b0a11f414fb42c534db715ef4d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.785883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb824b0a11f414fb42c534db715ef4d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.800202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb824b0a11f414fb42c534db715ef4d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.809224 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb824b0a11f414fb42c534db715ef4d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.809516 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-07d7c07a-da92-4f6f-ba25-979769cef086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb824b0a11f414fb42c534db715ef4d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:32.817209 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1352/2699] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:32 2022] DELETE /identity/v3/users/acb824b0a11f414fb42c534db715ef4d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:33.167653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:33.169142 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:33.186353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5bb5f-791e-4285-a44f-6e96ff9de87d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:33.273036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:33.274093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.274466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.274833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] PATH_INFO: `/v3/projects/a7e8d4314e034287b3a6816d21847926` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.276263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e8d4314e034287b3a6816d21847926)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:33.278349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:33.347354 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e8d4314e034287b3a6816d21847926', 'request_id': 'req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:33.354501 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e8d4314e034287b3a6816d21847926', 'request_id': 'req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:33.361460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e8d4314e034287b3a6816d21847926', 'request_id': 'req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:33.376344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] The token cache is being invalidate because project a7e8d4314e034287b3a6816d21847926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:33.376634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:33.377328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1c7e9569-47c7-49f6-bbd7-af2b25dc1348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:33.378862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1348/2700] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:33 2022] DELETE /identity/v3/projects/a7e8d4314e034287b3a6816d21847926 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:33.383791 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037865d-c02d-4b15-99f6-f6b1288eeb64 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.384188 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037865d-c02d-4b15-99f6-f6b1288eeb64 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.384537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037865d-c02d-4b15-99f6-f6b1288eeb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.406858 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-0037865d-c02d-4b15-99f6-f6b1288eeb64 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:33.408232 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-0037865d-c02d-4b15-99f6-f6b1288eeb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:33.472306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1353/2701] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:33.481164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a56d6c-0cc9-48f5-8b5b-bd49ecff2a9e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.482019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a56d6c-0cc9-48f5-8b5b-bd49ecff2a9e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.482854 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a56d6c-0cc9-48f5-8b5b-bd49ecff2a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.508031 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-16a56d6c-0cc9-48f5-8b5b-bd49ecff2a9e None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:33.509318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-16a56d6c-0cc9-48f5-8b5b-bd49ecff2a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:33.517870 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1349/2702] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:33.522736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037865d-c02d-4b15-99f6-f6b1288eeb64 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:33.533345 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037865d-c02d-4b15-99f6-f6b1288eeb64 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:33.538538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:33.539220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.539501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.539774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.540424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:07:33.540745 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:33.542360 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:33.557196 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1354/2703] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:07:33.562288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a56d6c-0cc9-48f5-8b5b-bd49ecff2a9e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:33.574738 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a56d6c-0cc9-48f5-8b5b-bd49ecff2a9e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:33.579840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:33.580457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.580761 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.581045 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.581722 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:33.583634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:33.641636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1350/2704] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:33 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:33.645477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:33.659416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f051c1-1b59-4e48-9025-c473f12cdc1f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:33.774832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:33.776789 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.777278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.777769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.778611 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:33.782207 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:33.795282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:33.824627 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1355/2705] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:33 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:07:33.825965 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:33.839112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb1d3-f1bb-499d-8a3d-202cc529fa08 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:33.845610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:33.846599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.847151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.847704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.848699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:33.850795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:33.864765 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1351/2706] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:33.868562 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:33.880589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b733-071d-4187-8716-6ae4f351e5e6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:33.886420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93b19e-d246-46a2-b827-88a083cfe16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:33.887132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93b19e-d246-46a2-b827-88a083cfe16d admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.887467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93b19e-d246-46a2-b827-88a083cfe16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.887810 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93b19e-d246-46a2-b827-88a083cfe16d admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.889588 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93b19e-d246-46a2-b827-88a083cfe16d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:33.891675 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93b19e-d246-46a2-b827-88a083cfe16d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:33.904518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1356/2707] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:33.908351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:33.921996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdd2c9-c665-4738-8a06-54f457b49c02 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:33.928413 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:33.929266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.929646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.930020 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] PATH_INFO: `/v3/projects/f33a45d9e04f4c07a075899f3be7283b/users/e05623bae2a044f1bce171169719abeb/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:33.949606 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33a45d9e04f4c07a075899f3be7283b, user_id=e05623bae2a044f1bce171169719abeb, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:33.953170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:33.957383 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:33.971033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1352/2708] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:33 2022] PUT /identity/v3/projects/f33a45d9e04f4c07a075899f3be7283b/users/e05623bae2a044f1bce171169719abeb/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:33.981315 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc14058-04f4-4659-b155-7a72cdaff8e0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:33.981775 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc14058-04f4-4659-b155-7a72cdaff8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:33.982784 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc14058-04f4-4659-b155-7a72cdaff8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.014677 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-abc14058-04f4-4659-b155-7a72cdaff8e0 None None] MFA Rules not processed for user `e05623bae2a044f1bce171169719abeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:34.016246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-abc14058-04f4-4659-b155-7a72cdaff8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:34.095109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1357/2709] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:34.096262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:34.192000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be858d-83f5-4a75-89ae-91dc86ea60d3 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:34.197967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:34.198782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.199230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.199871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.201332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:34.203491 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:34.254484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1353/2710] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:34 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:34.255662 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc14058-04f4-4659-b155-7a72cdaff8e0 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:34.267928 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc14058-04f4-4659-b155-7a72cdaff8e0 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:34.333265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:34.334043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.334404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.334786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.335759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:34.337809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:34.366835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:34.368939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1358/2711] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:34 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:07:34.378312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727e63c-680a-42fd-99a4-bec582f35f75 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:34.383266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:34.384072 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.384506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.385071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.385878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:34.387670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:34.403789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1354/2712] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:34.406061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:34.418042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe53e97-c8f0-457a-8ab5-004e4b39f41e admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:34.423787 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:34.424461 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.424774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.425053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] PATH_INFO: `/v3/projects/ad96dc16c23d446f962d02c633cc5610/users/3708851aaa244ae488ba9b093656bd6d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.442923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad96dc16c23d446f962d02c633cc5610, user_id=3708851aaa244ae488ba9b093656bd6d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:34.445780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:34.467940 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1359/2713] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:34 2022] PUT /identity/v3/projects/ad96dc16c23d446f962d02c633cc5610/users/3708851aaa244ae488ba9b093656bd6d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:34.468181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:34.487728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edfd71-4746-4710-9d9c-1e9ad41c13ca admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:34.559826 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b526b-d61b-4ef8-88d1-65bb2d82f4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:34.561240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b526b-d61b-4ef8-88d1-65bb2d82f4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.561513 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b526b-d61b-4ef8-88d1-65bb2d82f4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.561789 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b526b-d61b-4ef8-88d1-65bb2d82f4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.562450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b526b-d61b-4ef8-88d1-65bb2d82f4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:34.564137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b526b-d61b-4ef8-88d1-65bb2d82f4d6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:34.577535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1355/2714] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:34.581450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:34.592660 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346dfb8-0577-44cf-8b70-415a7f88a654 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:34.597498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:34.598237 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.598577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.598932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] PATH_INFO: `/v3/projects/ad96dc16c23d446f962d02c633cc5610/users/3708851aaa244ae488ba9b093656bd6d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.601008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad96dc16c23d446f962d02c633cc5610, user_id=3708851aaa244ae488ba9b093656bd6d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:34.603740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:34.634544 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1360/2715] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:34 2022] PUT /identity/v3/projects/ad96dc16c23d446f962d02c633cc5610/users/3708851aaa244ae488ba9b093656bd6d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:34.645959 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ae0e-7f83-4b85-a209-377f116b16f9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.646395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ae0e-7f83-4b85-a209-377f116b16f9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.646696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618ae0e-7f83-4b85-a209-377f116b16f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.679133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-5618ae0e-7f83-4b85-a209-377f116b16f9 None None] MFA Rules not processed for user `3708851aaa244ae488ba9b093656bd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:34.680369 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-5618ae0e-7f83-4b85-a209-377f116b16f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:34.742753 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:34.765283 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1356/2716] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:34.778721 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:34.780532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:34.804754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:34.859290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238c03c-3553-4210-a918-bbde5784527f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:34.865900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ba2c0-a5ee-4265-93a4-5cae54c3cfe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:34.866523 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ba2c0-a5ee-4265-93a4-5cae54c3cfe4 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:34.866797 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ba2c0-a5ee-4265-93a4-5cae54c3cfe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:34.867061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ba2c0-a5ee-4265-93a4-5cae54c3cfe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:34.877854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ba2c0-a5ee-4265-93a4-5cae54c3cfe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:34.879533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ba2c0-a5ee-4265-93a4-5cae54c3cfe4 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:34.898543 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1361/2717] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:07:35.694656 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ae0e-7f83-4b85-a209-377f116b16f9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:35.758230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618ae0e-7f83-4b85-a209-377f116b16f9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:35.762741 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:35.764389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:35.764673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:35.764980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] PATH_INFO: `/v3/users/e05623bae2a044f1bce171169719abeb` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:35.766104 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05623bae2a044f1bce171169719abeb)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:35.768040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:35.813465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05623bae2a044f1bce171169719abeb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:35.814114 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05623bae2a044f1bce171169719abeb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:35.820951 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05623bae2a044f1bce171169719abeb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:35.833949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05623bae2a044f1bce171169719abeb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:35.841013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05623bae2a044f1bce171169719abeb'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:35.847865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1357/2718] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:35 2022] DELETE /identity/v3/users/e05623bae2a044f1bce171169719abeb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:35.852260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:35.852778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:35.852880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:35.871586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:35.872863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:35.930178 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1362/2719] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:35.939846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:35.941340 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:35.998819 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b46a5-509e-4be9-982f-de8ca20323c5 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:36.002489 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:36.003078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:36.003349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:36.003613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:36.012428 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:36.014466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:36.023775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1358/2720] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:07:36.612624 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:36.614433 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:36.637846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ac34f-b09f-43a5-b7c8-19cfb7413b6c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:36.647105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:36.648466 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:36.648992 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:36.649568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] PATH_INFO: `/v3/projects/f33a45d9e04f4c07a075899f3be7283b` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:36.650915 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33a45d9e04f4c07a075899f3be7283b)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:36.662709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:36.738101 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33a45d9e04f4c07a075899f3be7283b', 'request_id': 'req-293a8463-1da0-44d7-8e11-b015550da320'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.746324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33a45d9e04f4c07a075899f3be7283b', 'request_id': 'req-293a8463-1da0-44d7-8e11-b015550da320'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.753055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33a45d9e04f4c07a075899f3be7283b', 'request_id': 'req-293a8463-1da0-44d7-8e11-b015550da320'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.765130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] The token cache is being invalidate because project f33a45d9e04f4c07a075899f3be7283b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:36.765415 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.766087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.767801 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1363/2721] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:36 2022] DELETE /identity/v3/projects/f33a45d9e04f4c07a075899f3be7283b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:36.771691 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:36.781639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d8aab-71dd-4cbc-854f-8561a1bb259b service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:36.791619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:36.848915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:36.849542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:36.849817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:36.850085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] PATH_INFO: `/v3/users/3708851aaa244ae488ba9b093656bd6d` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:36.851200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3708851aaa244ae488ba9b093656bd6d)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:36.853174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:36.902829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3708851aaa244ae488ba9b093656bd6d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.903718 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3708851aaa244ae488ba9b093656bd6d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.913898 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3708851aaa244ae488ba9b093656bd6d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.927305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3708851aaa244ae488ba9b093656bd6d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.934076 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-e9b1a1ab-90c8-47f8-a2c0-5f3ef490dba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3708851aaa244ae488ba9b093656bd6d'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:36.940859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1359/2722] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:36 2022] DELETE /identity/v3/users/3708851aaa244ae488ba9b093656bd6d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:07:37.408974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:37.411127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:37.428416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a8463-1da0-44d7-8e11-b015550da320 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:37.507557 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:37.512480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:37.514381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:37.514850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:37.515391 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] PATH_INFO: `/v3/projects/ad96dc16c23d446f962d02c633cc5610` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:37.517141 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad96dc16c23d446f962d02c633cc5610)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:37.519143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:37.572043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad96dc16c23d446f962d02c633cc5610', 'request_id': 'req-06715786-8530-4bdf-83b3-ec1b53925a4e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:37.581134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad96dc16c23d446f962d02c633cc5610', 'request_id': 'req-06715786-8530-4bdf-83b3-ec1b53925a4e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:37.590022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad96dc16c23d446f962d02c633cc5610', 'request_id': 'req-06715786-8530-4bdf-83b3-ec1b53925a4e'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:37.602317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] The token cache is being invalidate because project ad96dc16c23d446f962d02c633cc5610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:37.602621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:37.603248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-06715786-8530-4bdf-83b3-ec1b53925a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:37.605032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1364/2723] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:37 2022] DELETE /identity/v3/projects/ad96dc16c23d446f962d02c633cc5610 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:37.610702 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c77c4-affd-404d-b264-9b6364570e52 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:37.610988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c77c4-affd-404d-b264-9b6364570e52 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:37.611266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c77c4-affd-404d-b264-9b6364570e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:37.633887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1b7c77c4-affd-404d-b264-9b6364570e52 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:37.645135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1b7c77c4-affd-404d-b264-9b6364570e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:37.758953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1360/2724] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:37.769424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe5014-7ac2-457f-abff-1de75ed307bf None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:37.770909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe5014-7ac2-457f-abff-1de75ed307bf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:37.771108 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe5014-7ac2-457f-abff-1de75ed307bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:37.795328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-87fe5014-7ac2-457f-abff-1de75ed307bf None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:37.796696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-87fe5014-7ac2-457f-abff-1de75ed307bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:37.805852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1365/2725] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:37.810748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c77c4-affd-404d-b264-9b6364570e52 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:37.820699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c77c4-affd-404d-b264-9b6364570e52 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:37.825541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:37.826805 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:37.827146 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:37.827418 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:37.828062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81367) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:07:37.828359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:37.830002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:37.846002 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1361/2726] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:07:37.847248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe5014-7ac2-457f-abff-1de75ed307bf None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:37.856897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe5014-7ac2-457f-abff-1de75ed307bf None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:37.861724 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:37.862406 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:37.862754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:37.863092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:37.863872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:37.865906 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:37.899711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1366/2727] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:37 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:37.900953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:37.911164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3e51c-7fa5-41ee-bdcd-f182669b66b4 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:37.930612 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:37.975001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:37.975617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:37.975896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:37.976166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:37.976831 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:37.978766 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.006987 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1362/2728] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:37 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:38.008474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.019727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86306c-2256-4a77-afca-2df42bd7b659 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.024373 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.025074 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.025339 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.025600 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.026296 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.027883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.039111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1367/2729] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:38.045176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.059109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a47a3-f07a-4b3f-aee5-006300436b92 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.065905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3edfa24-c79e-4087-889c-5a03fbdbb5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.067622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edfa24-c79e-4087-889c-5a03fbdbb5fa admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.067932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edfa24-c79e-4087-889c-5a03fbdbb5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.068209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3edfa24-c79e-4087-889c-5a03fbdbb5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.068895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edfa24-c79e-4087-889c-5a03fbdbb5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.071179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3edfa24-c79e-4087-889c-5a03fbdbb5fa admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.087260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1363/2730] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:38.091384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.101880 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff2add-3515-461b-b00e-f236994bf493 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.112022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.113274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.113274 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.113464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] PATH_INFO: `/v3/projects/d83c9b7909e44177a0478444664ea63c/users/bb4dd3eb61d7409cbac2b9180f8b5b0f/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.131852 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83c9b7909e44177a0478444664ea63c, user_id=bb4dd3eb61d7409cbac2b9180f8b5b0f, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.134541 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.148581 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1368/2731] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:38 2022] PUT /identity/v3/projects/d83c9b7909e44177a0478444664ea63c/users/bb4dd3eb61d7409cbac2b9180f8b5b0f/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:38.158924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd252a11-b4b3-4bc6-a489-4d2bb4a568d8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.159209 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd252a11-b4b3-4bc6-a489-4d2bb4a568d8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.159475 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd252a11-b4b3-4bc6-a489-4d2bb4a568d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.191320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-cd252a11-b4b3-4bc6-a489-4d2bb4a568d8 None None] MFA Rules not processed for user `bb4dd3eb61d7409cbac2b9180f8b5b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:38.192553 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-cd252a11-b4b3-4bc6-a489-4d2bb4a568d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:38.262588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1364/2732] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:38.267060 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.337131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e5fa1-c95e-49ae-bd50-70e59d608856 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.342362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.342968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.343243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.343518 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.344153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.346105 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.369275 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:38.381430 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1369/2733] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:38 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:07:38.382422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd252a11-b4b3-4bc6-a489-4d2bb4a568d8 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.393165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd252a11-b4b3-4bc6-a489-4d2bb4a568d8 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.448685 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.449564 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.449843 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.450110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.450773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.452697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.474281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1365/2734] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:38 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:38.478218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.489151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77cdfcc-35d2-4386-bae2-759fd6cbf728 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.493802 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.494388 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.494671 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.494939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.495573 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.497215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.507597 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1370/2735] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:38.511631 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.522239 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c0dcc-616b-4b52-a8cb-33582fed39e0 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.527000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.527653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.528047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.528328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] PATH_INFO: `/v3/projects/702527b37a87491393931b0ca7974f0f/users/875388c66f494ca0a05328d13eabc131/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.543840 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702527b37a87491393931b0ca7974f0f, user_id=875388c66f494ca0a05328d13eabc131, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.547299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.549675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:38.565153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1366/2736] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:38 2022] PUT /identity/v3/projects/702527b37a87491393931b0ca7974f0f/users/875388c66f494ca0a05328d13eabc131/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:38.569474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.580160 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dfce4-4b16-4636-a90e-171d4294e7d7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.643895 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98216000-2f6f-4e54-bbb9-7917e508f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.644522 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98216000-2f6f-4e54-bbb9-7917e508f204 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.644836 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98216000-2f6f-4e54-bbb9-7917e508f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.645113 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98216000-2f6f-4e54-bbb9-7917e508f204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.645768 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98216000-2f6f-4e54-bbb9-7917e508f204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.647396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98216000-2f6f-4e54-bbb9-7917e508f204 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.661608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1371/2737] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:38.665919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.679351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8675-220d-40c2-91b3-6ac9ab225ea6 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.686164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.687129 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.687548 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.687910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] PATH_INFO: `/v3/projects/702527b37a87491393931b0ca7974f0f/users/875388c66f494ca0a05328d13eabc131/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.689916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702527b37a87491393931b0ca7974f0f, user_id=875388c66f494ca0a05328d13eabc131, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:38.693466 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:38.708849 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1367/2738] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:38 2022] PUT /identity/v3/projects/702527b37a87491393931b0ca7974f0f/users/875388c66f494ca0a05328d13eabc131/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:38.723325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.723325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.723325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:38.757970 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] MFA Rules not processed for user `875388c66f494ca0a05328d13eabc131`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:38.759331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:38.854033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1372/2739] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:07:38.865262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:38.867057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:38.934227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac801b-c4ed-4da5-a328-86a161614f23 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:38.938179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:38.938859 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:38.939197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:38.939531 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:39.018001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:39.020696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:39.046597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1368/2740] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:07:43.379233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:43.380613 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:43.393246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f2b9f-c061-45e8-b4bd-6dd6cd80d68c None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:43.396686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:43.397329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:43.397606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:43.397871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:43.407140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:43.408556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:43.419214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1373/2741] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:44.325745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:44.327486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:44.383665 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97623e1-c9ce-41d1-a801-8f510535b36e service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:44.387232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b32443-c104-4aad-bebb-29312ec623cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:44.387855 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b32443-c104-4aad-bebb-29312ec623cc service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:44.388120 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b32443-c104-4aad-bebb-29312ec623cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:44.388439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b32443-c104-4aad-bebb-29312ec623cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:44.390180 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-46b32443-c104-4aad-bebb-29312ec623cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:44.436999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:44.452757 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b32443-c104-4aad-bebb-29312ec623cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:44.454203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b32443-c104-4aad-bebb-29312ec623cc service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:44.463803 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1369/2742] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:48.500116 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:48.561336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:48.567506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649c85-30e5-4e6e-9730-6636ae5fa286 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:48.573574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:48.574202 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:48.574477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:48.574746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] PATH_INFO: `/v3/users/bb4dd3eb61d7409cbac2b9180f8b5b0f` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:48.576820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4dd3eb61d7409cbac2b9180f8b5b0f)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:48.578741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:48.626995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4dd3eb61d7409cbac2b9180f8b5b0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:48.634524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4dd3eb61d7409cbac2b9180f8b5b0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:48.648158 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4dd3eb61d7409cbac2b9180f8b5b0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:48.656025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4dd3eb61d7409cbac2b9180f8b5b0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:48.656746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4dd3eb61d7409cbac2b9180f8b5b0f'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:48.662796 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1374/2743] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:48 2022] DELETE /identity/v3/users/bb4dd3eb61d7409cbac2b9180f8b5b0f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:48.667541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:48.668037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:48.668137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:48.693276 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:48.694567 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:48.770682 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1370/2744] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:48.783091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:48.784556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:48.865497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef49b43-67ef-47d7-b6d9-4c33c720ba2b admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:48.871055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:48.871668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:48.872485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:48.872779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:48.884546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:48.886028 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:48.908829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1375/2745] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:49.224756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:49.226424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:49.252637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3a9bb-4758-471e-a8a0-ddf53df5fc26 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:49.259121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:49.260067 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:49.260580 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:49.261063 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] PATH_INFO: `/v3/projects/d83c9b7909e44177a0478444664ea63c` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:49.263177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83c9b7909e44177a0478444664ea63c)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:49.265887 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:49.317946 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83c9b7909e44177a0478444664ea63c', 'request_id': 'req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.326154 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83c9b7909e44177a0478444664ea63c', 'request_id': 'req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.332172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83c9b7909e44177a0478444664ea63c', 'request_id': 'req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.344074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] The token cache is being invalidate because project d83c9b7909e44177a0478444664ea63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:49.344492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.345252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.346851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1371/2746] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:49 2022] DELETE /identity/v3/projects/d83c9b7909e44177a0478444664ea63c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:49.350847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:49.361152 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1d7a5-7c4b-474b-a22d-54101cf7d10f service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:49.418567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:49.419723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:49.420067 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:49.420341 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] PATH_INFO: `/v3/users/875388c66f494ca0a05328d13eabc131` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:49.421530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] RBAC: Authorizing `identity:delete_user(user_id=875388c66f494ca0a05328d13eabc131)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:49.423425 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:49.446241 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:49.473070 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875388c66f494ca0a05328d13eabc131'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.482616 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875388c66f494ca0a05328d13eabc131'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.495818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875388c66f494ca0a05328d13eabc131'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.503476 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875388c66f494ca0a05328d13eabc131'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.504120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-f6b4de32-9ef3-4762-9a75-1e0fbed5c62b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875388c66f494ca0a05328d13eabc131'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.510853 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1376/2747] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:49 2022] DELETE /identity/v3/users/875388c66f494ca0a05328d13eabc131 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:49.784051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:49.785642 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:49.802298 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bd7db-3ef1-4fc3-b721-0f9d6598e945 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:49.869064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:49.869605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:49.869878 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:49.870145 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] PATH_INFO: `/v3/projects/702527b37a87491393931b0ca7974f0f` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:49.871252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=702527b37a87491393931b0ca7974f0f)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:49.874497 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:49.885192 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:49.929542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702527b37a87491393931b0ca7974f0f', 'request_id': 'req-d96cae80-38af-4fb4-935d-7d7bad820b44'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.939680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702527b37a87491393931b0ca7974f0f', 'request_id': 'req-d96cae80-38af-4fb4-935d-7d7bad820b44'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.950599 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702527b37a87491393931b0ca7974f0f', 'request_id': 'req-d96cae80-38af-4fb4-935d-7d7bad820b44'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.979086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] The token cache is being invalidate because project 702527b37a87491393931b0ca7974f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:49.979451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.980245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-d96cae80-38af-4fb4-935d-7d7bad820b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:49.984768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1372/2748] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:49 2022] DELETE /identity/v3/projects/702527b37a87491393931b0ca7974f0f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:49.988381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380564f-f220-442d-b67a-5d92c9abb9bb None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:49.988666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380564f-f220-442d-b67a-5d92c9abb9bb None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:49.988977 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380564f-f220-442d-b67a-5d92c9abb9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.010061 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d380564f-f220-442d-b67a-5d92c9abb9bb None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:50.011273 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d380564f-f220-442d-b67a-5d92c9abb9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:50.081441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1377/2749] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:50.090099 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92485cc4-7e16-4b7b-bf0c-eb4f561af8c2 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.090472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92485cc4-7e16-4b7b-bf0c-eb4f561af8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.090842 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92485cc4-7e16-4b7b-bf0c-eb4f561af8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.113169 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-92485cc4-7e16-4b7b-bf0c-eb4f561af8c2 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:50.114535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-92485cc4-7e16-4b7b-bf0c-eb4f561af8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:50.125431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1373/2750] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:50.129231 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380564f-f220-442d-b67a-5d92c9abb9bb None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.139948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380564f-f220-442d-b67a-5d92c9abb9bb None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.144698 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.145347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.145620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.145884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.146659 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:07:50.146968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.148589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.162416 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1378/2751] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:07:50.166533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92485cc4-7e16-4b7b-bf0c-eb4f561af8c2 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.176100 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92485cc4-7e16-4b7b-bf0c-eb4f561af8c2 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.180910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.181522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.181794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.182061 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.182709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.184756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.222127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1374/2752] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:50 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:50.223331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.236186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e16cd3-8119-4b4f-a7ff-879be9e6f9b5 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.303715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.304344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.304842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.305120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.305771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.307780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.336867 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1379/2753] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:50 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:07:50.341105 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.355241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7262cdb-3ae5-4e9f-9b5f-afdcbea53e7d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.360538 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.361223 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.361521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.361769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.362430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.364074 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.376403 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1375/2754] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:50.380398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.391529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e8152-16ee-432a-baaf-75380a649201 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.398330 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5753ac-f719-47c6-9599-7ed88ec29b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.399098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5753ac-f719-47c6-9599-7ed88ec29b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.399480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5753ac-f719-47c6-9599-7ed88ec29b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.399846 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5753ac-f719-47c6-9599-7ed88ec29b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.400602 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5753ac-f719-47c6-9599-7ed88ec29b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.402367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5753ac-f719-47c6-9599-7ed88ec29b07 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.414767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1380/2755] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:50.419604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.430777 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d4d9-155b-4cee-bc7b-886bf25b48af admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.436268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.436866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.437150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.437431 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] PATH_INFO: `/v3/projects/6d54213fc22d47eebc5178c7f7ee5922/users/ecf17d2edb67436690827d7e003233df/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.455996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d54213fc22d47eebc5178c7f7ee5922, user_id=ecf17d2edb67436690827d7e003233df, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.458886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.476411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1376/2756] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:50 2022] PUT /identity/v3/projects/6d54213fc22d47eebc5178c7f7ee5922/users/ecf17d2edb67436690827d7e003233df/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:50.487981 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef880f-6621-455d-841c-6009157cc0ab None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.488626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef880f-6621-455d-841c-6009157cc0ab None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.488993 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef880f-6621-455d-841c-6009157cc0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.497657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:50.523246 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-57ef880f-6621-455d-841c-6009157cc0ab None None] MFA Rules not processed for user `ecf17d2edb67436690827d7e003233df`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:50.524582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-57ef880f-6621-455d-841c-6009157cc0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:50.603964 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1381/2757] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:50.605845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.682329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ecb67-31d2-46c3-83dc-2e94690eaae0 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.687373 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.687989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.688270 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.688543 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.689211 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.691150 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.723245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1377/2758] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:50 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:50.725100 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef880f-6621-455d-841c-6009157cc0ab None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.742248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef880f-6621-455d-841c-6009157cc0ab None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.813404 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.814030 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.814310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.814579 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.815580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.817362 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.845888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1382/2759] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:50 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:50.847109 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.859258 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d2a69-a574-4e10-82fb-e904802fbee8 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.864477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.865207 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.865554 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.865892 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.866645 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.868411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.883471 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1378/2760] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:50.884674 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.895962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b73654-b635-4496-8f35-bb2d3ba75d10 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.901884 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be20cf-3d4c-4006-ab86-904518bbb1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.902503 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be20cf-3d4c-4006-ab86-904518bbb1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.902771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be20cf-3d4c-4006-ab86-904518bbb1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.903066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be20cf-3d4c-4006-ab86-904518bbb1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.903731 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be20cf-3d4c-4006-ab86-904518bbb1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.905932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be20cf-3d4c-4006-ab86-904518bbb1c1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.920289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1383/2761] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:07:50.921675 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:50.933191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a923afa-1b86-4ea7-a288-ec92cae8370d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:50.941485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:50.942266 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:50.942616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:50.942961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] PATH_INFO: `/v3/projects/8aefd5a642384d8f9808600abdc7ed7f/users/400d55a7469f478980d4dcc22529b6c7/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:50.947576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:50.963286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aefd5a642384d8f9808600abdc7ed7f, user_id=400d55a7469f478980d4dcc22529b6c7, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:50.966736 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:50.983182 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1379/2762] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:50 2022] PUT /identity/v3/projects/8aefd5a642384d8f9808600abdc7ed7f/users/400d55a7469f478980d4dcc22529b6c7/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:51.057567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a44f34-13c3-4b9c-93f3-8deb89285847 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.057842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a44f34-13c3-4b9c-93f3-8deb89285847 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.058110 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a44f34-13c3-4b9c-93f3-8deb89285847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.088827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-d5a44f34-13c3-4b9c-93f3-8deb89285847 None None] MFA Rules not processed for user `400d55a7469f478980d4dcc22529b6c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:51.090044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d5a44f34-13c3-4b9c-93f3-8deb89285847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:51.156449 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:51.160532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1384/2763] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:51.164844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:51.233281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2449-9cd6-4c2d-bc40-58c8623992ab admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:51.238177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:51.238801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.239081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.239353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.240003 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:51.241994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:51.270908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1380/2764] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:07:51 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:07:51.275222 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a44f34-13c3-4b9c-93f3-8deb89285847 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:51.287556 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a44f34-13c3-4b9c-93f3-8deb89285847 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:51.364702 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:51.365811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.365995 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.366357 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.367225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:51.369355 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:51.394481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1385/2765] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:07:51 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:07:51.398398 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:51.410253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d688b1d-16f3-4360-9afb-a5b2fc032fea admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:51.414838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:51.415437 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.415716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.416148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.416844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:51.418457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:51.430167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1381/2766] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:07:51.434214 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:51.444689 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eed915-955f-4841-876c-ca8d34fd5d84 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:51.449338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:51.450529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.450809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.451083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] PATH_INFO: `/v3/projects/7cbc2be0ee9645e6b002069426709642/users/921326c4982e484b801f5f8d5798fb2a/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.467967 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbc2be0ee9645e6b002069426709642, user_id=921326c4982e484b801f5f8d5798fb2a, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:51.470657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:51.483057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1386/2767] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:51 2022] PUT /identity/v3/projects/7cbc2be0ee9645e6b002069426709642/users/921326c4982e484b801f5f8d5798fb2a/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:51.486967 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:51.496949 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4954d-b4de-4635-a07a-50506053d30d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:51.553130 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7674ee-2d3f-482d-8c0b-a5c375aa286e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:51.553727 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7674ee-2d3f-482d-8c0b-a5c375aa286e admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.553998 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7674ee-2d3f-482d-8c0b-a5c375aa286e admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.554265 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7674ee-2d3f-482d-8c0b-a5c375aa286e admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.554896 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7674ee-2d3f-482d-8c0b-a5c375aa286e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:51.556484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7674ee-2d3f-482d-8c0b-a5c375aa286e admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:51.566221 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1382/2768] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:07:51.570511 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:51.581151 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310fa90-975e-4882-b11a-f1395b78981f admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:51.585877 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:51.586506 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.586779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.587043 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] PATH_INFO: `/v3/projects/7cbc2be0ee9645e6b002069426709642/users/921326c4982e484b801f5f8d5798fb2a/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.588948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbc2be0ee9645e6b002069426709642, user_id=921326c4982e484b801f5f8d5798fb2a, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:51.592384 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:51.602379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1387/2769] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:07:51 2022] PUT /identity/v3/projects/7cbc2be0ee9645e6b002069426709642/users/921326c4982e484b801f5f8d5798fb2a/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:51.612787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.613286 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.613799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.644895 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] MFA Rules not processed for user `921326c4982e484b801f5f8d5798fb2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:51.646214 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:51.690093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:51.717046 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1383/2770] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:51.728495 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:51.730267 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:51.787532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb60589-7a7d-4f27-b54d-2cadb0f62362 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:51.791271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:51.791866 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:51.792137 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:51.792401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:51.848439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:51.850481 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:51.860325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1388/2771] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:53.359780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:53.361256 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:53.373977 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918795e-26af-4514-bbbe-03b154acd8e9 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:53.377515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:53.378107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:53.378379 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:53.378644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:53.434173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:53.435570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:53.445701 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1384/2772] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:53.955769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:53.964855 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410f6e7-e0f0-4811-b981-bb527da37193 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:53.969173 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:53.969766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:53.970042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:53.970310 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:53.976016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:53.980948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:53.982949 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:53.992626 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1389/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:55.612526 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:55.613986 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:55.674138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc31a5-096f-4526-a39d-206b108787f1 service cinder] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:55.677698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7129ab-b3ee-4038-9c4b-27568657d424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:55.678299 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7129ab-b3ee-4038-9c4b-27568657d424 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:55.678576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7129ab-b3ee-4038-9c4b-27568657d424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:55.678845 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7129ab-b3ee-4038-9c4b-27568657d424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:55.680583 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-dd7129ab-b3ee-4038-9c4b-27568657d424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:55.742591 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7129ab-b3ee-4038-9c4b-27568657d424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:55.744027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7129ab-b3ee-4038-9c4b-27568657d424 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:55.754603 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1385/2774] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:57.941553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:58.019818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b482b0-dc39-4506-a689-95685c2ec6ef service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:58.027486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:58.028435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:58.028714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:58.029008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] PATH_INFO: `/v3/users/ecf17d2edb67436690827d7e003233df` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:58.030953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf17d2edb67436690827d7e003233df)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:58.033959 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:58.115780 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf17d2edb67436690827d7e003233df'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:58.125215 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf17d2edb67436690827d7e003233df'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:58.140210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf17d2edb67436690827d7e003233df'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:58.150126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf17d2edb67436690827d7e003233df'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:58.150953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf17d2edb67436690827d7e003233df'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:58.162917 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1390/2775] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:57 2022] DELETE /identity/v3/users/ecf17d2edb67436690827d7e003233df => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:58.167395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:58.167885 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:58.167983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:58.188278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:07:58.189458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:58.232282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:58.279196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1386/2776] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:07:58.290018 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:58.291520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:58.367699 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2518b-ae60-4d8f-b676-66a779778e7b admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:58.371818 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:58.372767 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:58.372913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:58.373239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:58.386293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:58.389564 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:58.403261 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1391/2777] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:07:58.927460 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:58.929357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:58.944646 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec46d8-f67e-4752-924f-ea3c758b696c None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:58.950521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:58.952487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:58.952487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:58.952743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] PATH_INFO: `/v3/projects/6d54213fc22d47eebc5178c7f7ee5922` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:58.954057 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d54213fc22d47eebc5178c7f7ee5922)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:58.956181 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:59.021485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d54213fc22d47eebc5178c7f7ee5922', 'request_id': 'req-43735598-ae1e-47c8-a193-5c9e8a44ecc8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.029458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d54213fc22d47eebc5178c7f7ee5922', 'request_id': 'req-43735598-ae1e-47c8-a193-5c9e8a44ecc8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.039940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d54213fc22d47eebc5178c7f7ee5922', 'request_id': 'req-43735598-ae1e-47c8-a193-5c9e8a44ecc8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.053697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] The token cache is being invalidate because project 6d54213fc22d47eebc5178c7f7ee5922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:59.053980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.054650 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.056164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1387/2778] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:58 2022] DELETE /identity/v3/projects/6d54213fc22d47eebc5178c7f7ee5922 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:59.060289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:59.072393 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f15eec-34ed-447b-8a18-59b7624ed291 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:59.085687 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:59.137939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:59.138592 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:59.139022 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:59.139480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] PATH_INFO: `/v3/users/400d55a7469f478980d4dcc22529b6c7` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:59.140680 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=400d55a7469f478980d4dcc22529b6c7)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:59.142783 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:59.197520 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400d55a7469f478980d4dcc22529b6c7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.208375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400d55a7469f478980d4dcc22529b6c7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.225255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400d55a7469f478980d4dcc22529b6c7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.235443 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400d55a7469f478980d4dcc22529b6c7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.236181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400d55a7469f478980d4dcc22529b6c7'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.242524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1392/2779] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:59 2022] DELETE /identity/v3/users/400d55a7469f478980d4dcc22529b6c7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:59.743584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:59.745168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:59.760680 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43735598-ae1e-47c8-a193-5c9e8a44ecc8 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:07:59.838430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:07:59.839235 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:07:59.839576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:07:59.839916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] PATH_INFO: `/v3/projects/8aefd5a642384d8f9808600abdc7ed7f` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:07:59.841191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aefd5a642384d8f9808600abdc7ed7f)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:07:59.843393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:07:59.899529 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aefd5a642384d8f9808600abdc7ed7f', 'request_id': 'req-a81700cd-861d-4987-9c6a-3511b68c0702'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.903934 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:07:59.911635 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aefd5a642384d8f9808600abdc7ed7f', 'request_id': 'req-a81700cd-861d-4987-9c6a-3511b68c0702'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.922064 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aefd5a642384d8f9808600abdc7ed7f', 'request_id': 'req-a81700cd-861d-4987-9c6a-3511b68c0702'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.934725 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] The token cache is being invalidate because project 8aefd5a642384d8f9808600abdc7ed7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:07:59.935056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.935689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:07:59.940865 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1388/2780] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:07:59 2022] DELETE /identity/v3/projects/8aefd5a642384d8f9808600abdc7ed7f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:07:59.941979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:07:59.943424 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:07:59.959826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6112924-98a1-46f1-9e72-bd43f7124fe9 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.018143 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.018778 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.019097 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.019320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] PATH_INFO: `/v3/users/921326c4982e484b801f5f8d5798fb2a` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.020454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=921326c4982e484b801f5f8d5798fb2a)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.022572 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.063431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:00.073065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921326c4982e484b801f5f8d5798fb2a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.081377 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921326c4982e484b801f5f8d5798fb2a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.098776 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921326c4982e484b801f5f8d5798fb2a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.107011 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921326c4982e484b801f5f8d5798fb2a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.108554 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2c79cdd9-580d-49aa-b40b-b43bb6cc95bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921326c4982e484b801f5f8d5798fb2a'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.115726 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1393/2781] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:07:59 2022] DELETE /identity/v3/users/921326c4982e484b801f5f8d5798fb2a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:08:00.406594 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:00.408135 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:00.422112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81700cd-861d-4987-9c6a-3511b68c0702 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.486637 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.487268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.487574 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.487848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] PATH_INFO: `/v3/projects/7cbc2be0ee9645e6b002069426709642` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.489028 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbc2be0ee9645e6b002069426709642)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.491015 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.536000 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbc2be0ee9645e6b002069426709642', 'request_id': 'req-53d321e3-4b4f-4712-85ef-d226e58cc223'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.543368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbc2be0ee9645e6b002069426709642', 'request_id': 'req-53d321e3-4b4f-4712-85ef-d226e58cc223'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.549871 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbc2be0ee9645e6b002069426709642', 'request_id': 'req-53d321e3-4b4f-4712-85ef-d226e58cc223'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.561940 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] The token cache is being invalidate because project 7cbc2be0ee9645e6b002069426709642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:08:00.562226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.562884 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-53d321e3-4b4f-4712-85ef-d226e58cc223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:00.564361 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1389/2782] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:08:00 2022] DELETE /identity/v3/projects/7cbc2be0ee9645e6b002069426709642 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:08:00.569349 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7364a-431b-47e5-a3a5-a3b55da5e6fd None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.569575 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7364a-431b-47e5-a3a5-a3b55da5e6fd None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.569849 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7364a-431b-47e5-a3a5-a3b55da5e6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.589643 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bbb7364a-431b-47e5-a3a5-a3b55da5e6fd None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:00.590837 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bbb7364a-431b-47e5-a3a5-a3b55da5e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:00.653112 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1394/2783] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:08:00.662565 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54074c11-ac05-498a-a12d-a8a7631b9133 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.662875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54074c11-ac05-498a-a12d-a8a7631b9133 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.663172 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54074c11-ac05-498a-a12d-a8a7631b9133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.683252 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-54074c11-ac05-498a-a12d-a8a7631b9133 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:00.684450 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-54074c11-ac05-498a-a12d-a8a7631b9133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:00.693033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1390/2784] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:08:00.697302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7364a-431b-47e5-a3a5-a3b55da5e6fd None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:00.707270 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7364a-431b-47e5-a3a5-a3b55da5e6fd None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.712013 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.712601 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.712904 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.713172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.713832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:08:00.714130 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.715711 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.727723 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1395/2785] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:08:00.732170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54074c11-ac05-498a-a12d-a8a7631b9133 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:00.745769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54074c11-ac05-498a-a12d-a8a7631b9133 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.750454 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.751051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.751330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.751598 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.752237 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.754165 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.781740 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1391/2786] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:08:00 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:08:00.785876 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:00.798823 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16dc8a8-2590-461c-8aff-ec336a9ace82 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.854870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.855396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.855669 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.855938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.856563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.859037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.879781 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1396/2787] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:08:00 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:08:00.883923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:00.894051 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7db4f4-e524-4682-8e86-632c2af21f39 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.898610 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.899193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.899465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.899731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.900367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.901986 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.912149 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1392/2788] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:08:00.915923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:00.926487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92915f-d612-47f8-ab3e-db2a53aa0e04 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.931051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c46cfb-8324-4660-87e4-fc8cf937fc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.931632 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c46cfb-8324-4660-87e4-fc8cf937fc77 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.931907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c46cfb-8324-4660-87e4-fc8cf937fc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.932171 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c46cfb-8324-4660-87e4-fc8cf937fc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.932829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c46cfb-8324-4660-87e4-fc8cf937fc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.934403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c46cfb-8324-4660-87e4-fc8cf937fc77 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.944888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1397/2789] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:08:00.948834 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:00.958821 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1267fc0-6d6b-406c-849e-df23a86975e1 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:00.963536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:00.964167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:00.964438 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:00.964707 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] PATH_INFO: `/v3/projects/f63f9a22c695451da76e8794f1b6a7a9/users/3467cd26ac30452eb0e9004903313525/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:00.979414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63f9a22c695451da76e8794f1b6a7a9, user_id=3467cd26ac30452eb0e9004903313525, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:00.982098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:00.994782 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1393/2790] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:08:00 2022] PUT /identity/v3/projects/f63f9a22c695451da76e8794f1b6a7a9/users/3467cd26ac30452eb0e9004903313525/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:08:01.005124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581415b8-6887-45eb-b873-a8df914360a8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:01.005440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581415b8-6887-45eb-b873-a8df914360a8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:01.005715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581415b8-6887-45eb-b873-a8df914360a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:01.033590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-581415b8-6887-45eb-b873-a8df914360a8 None None] MFA Rules not processed for user `3467cd26ac30452eb0e9004903313525`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:01.034804 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-581415b8-6887-45eb-b873-a8df914360a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:01.043133 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:01.107779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1398/2791] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:08:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:08:01.118417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:01.120085 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:01.130983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:01.179103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8860c-5d9f-4729-b18c-812b1fe9771f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:01.182852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b85c3-c377-4bc3-9b8a-be103e792951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:01.183461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b85c3-c377-4bc3-9b8a-be103e792951 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:01.183742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b85c3-c377-4bc3-9b8a-be103e792951 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:01.184016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b85c3-c377-4bc3-9b8a-be103e792951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:01.192929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b85c3-c377-4bc3-9b8a-be103e792951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:01.194380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b85c3-c377-4bc3-9b8a-be103e792951 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:01.203795 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1394/2792] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:08:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:08:02.564538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:02.565084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:02.565181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:02.585129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] MFA Rules not processed for user `3467cd26ac30452eb0e9004903313525`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:02.586252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:02.593889 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1399/2793] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:08:11.199303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=22 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:12.580045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=26 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:17.743166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2bf6d-371e-4497-974e-b8040071beb9 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:17.743662 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2bf6d-371e-4497-974e-b8040071beb9 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:17.743776 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2bf6d-371e-4497-974e-b8040071beb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:17.763345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-9fb2bf6d-371e-4497-974e-b8040071beb9 None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:17.764573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9fb2bf6d-371e-4497-974e-b8040071beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:17.821401 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1395/2794] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:08:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:08:17.842803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:17.844221 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:17.912931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca784a-7e16-403f-9006-6a55f7faf086 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:17.917843 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:17.918435 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:17.918709 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:17.918979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:17.927785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:17.929387 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:17.938186 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1400/2795] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:08:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:08:18.431709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558969d1-7742-43d4-8012-9eeaebc33e4a None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:18.431993 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558969d1-7742-43d4-8012-9eeaebc33e4a None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:18.432268 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558969d1-7742-43d4-8012-9eeaebc33e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:18.452550 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-558969d1-7742-43d4-8012-9eeaebc33e4a None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:18.453781 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-558969d1-7742-43d4-8012-9eeaebc33e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:18.461101 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1396/2796] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 16:08:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:08:18.474177 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:18.485477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c16f0b-fe34-4d5b-8048-f0c773f19260 service swift] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:18.490667 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:18.491258 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:18.491553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:18.491819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:18.500683 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:18.502287 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:18.510190 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1401/2797] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:08:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:08:18.733392 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4e16-3ce4-403e-add9-84be56e27a4d None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:18.733678 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4e16-3ce4-403e-add9-84be56e27a4d None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:18.733995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4e16-3ce4-403e-add9-84be56e27a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:18.752982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-d48b4e16-3ce4-403e-add9-84be56e27a4d None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:18.754166 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-d48b4e16-3ce4-403e-add9-84be56e27a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:18.761148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1397/2798] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Oct 3 16:08:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:08:18.773118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:18.784195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d193934-1e23-4e9a-b4e7-d766a01f8d62 service swift] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:18.789181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:18.789770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:18.790048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:18.790317 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:18.799047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:18.800638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:18.808559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1402/2799] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:08:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:08:19.124882 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67f2b1-e15b-4216-af1e-852d5b1a8b2e None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:19.125170 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67f2b1-e15b-4216-af1e-852d5b1a8b2e None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:19.125445 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67f2b1-e15b-4216-af1e-852d5b1a8b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:19.145022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-0c67f2b1-e15b-4216-af1e-852d5b1a8b2e None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:19.146228 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-0c67f2b1-e15b-4216-af1e-852d5b1a8b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:19.153451 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1398/2800] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 16:08:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:08:19.165943 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:19.176863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569c767-6f47-4599-9e3f-26f6edaf0936 service swift] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:19.181814 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27b881-922e-4284-9f52-9e3d4fdb5489 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:19.182423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27b881-922e-4284-9f52-9e3d4fdb5489 service swift] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:19.182696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27b881-922e-4284-9f52-9e3d4fdb5489 service swift] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:19.182962 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27b881-922e-4284-9f52-9e3d4fdb5489 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:19.190894 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27b881-922e-4284-9f52-9e3d4fdb5489 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:19.192454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27b881-922e-4284-9f52-9e3d4fdb5489 service swift] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:19.200239 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1403/2801] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:08:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:08:21.286816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67f2b1-e15b-4216-af1e-852d5b1a8b2e None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:21.350943 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67f2b1-e15b-4216-af1e-852d5b1a8b2e None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:21.355472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:21.356090 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:21.356368 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:21.356640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] PATH_INFO: `/v3/users/3467cd26ac30452eb0e9004903313525` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:21.358816 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3467cd26ac30452eb0e9004903313525)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:21.359801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:21.408098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3467cd26ac30452eb0e9004903313525'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.408763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3467cd26ac30452eb0e9004903313525'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.415640 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3467cd26ac30452eb0e9004903313525'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.428152 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3467cd26ac30452eb0e9004903313525'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.434891 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3467cd26ac30452eb0e9004903313525'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.441541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1399/2802] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:08:21 2022] DELETE /identity/v3/users/3467cd26ac30452eb0e9004903313525 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:08:21.446134 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:21.446606 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:21.446705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:21.468363 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:21.469594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:21.530822 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1404/2803] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:08:21.540303 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:21.541735 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:21.548679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:21.602121 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f92-95a2-4416-9e85-b5715ccad4f9 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:21.605768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fef633-5c6c-4ad2-99f5-c80cb8d21f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:21.606395 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fef633-5c6c-4ad2-99f5-c80cb8d21f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:21.606674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fef633-5c6c-4ad2-99f5-c80cb8d21f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:21.606941 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fef633-5c6c-4ad2-99f5-c80cb8d21f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:21.616906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fef633-5c6c-4ad2-99f5-c80cb8d21f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:21.618345 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fef633-5c6c-4ad2-99f5-c80cb8d21f8e service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:21.628605 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1400/2804] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:08:21.794131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:08:21.796436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:21.811552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ee883-8f17-4b9b-b501-da1558f61d5d None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:08:21.817031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:08:21.817729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:21.818004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:21.818277 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] PATH_INFO: `/v3/projects/f63f9a22c695451da76e8794f1b6a7a9` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:21.819565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63f9a22c695451da76e8794f1b6a7a9)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:08:21.821942 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:08:21.869629 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:21.872561 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63f9a22c695451da76e8794f1b6a7a9', 'request_id': 'req-45d59cfa-789c-4e53-80be-562c59734640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.881153 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63f9a22c695451da76e8794f1b6a7a9', 'request_id': 'req-45d59cfa-789c-4e53-80be-562c59734640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.886728 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63f9a22c695451da76e8794f1b6a7a9', 'request_id': 'req-45d59cfa-789c-4e53-80be-562c59734640'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.898431 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] The token cache is being invalidate because project f63f9a22c695451da76e8794f1b6a7a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81368) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:08:21.898713 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.899337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:08:21.900826 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1405/2805] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:08:21 2022] DELETE /identity/v3/projects/f63f9a22c695451da76e8794f1b6a7a9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:08:31.624294 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=24 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:31.895563 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:08:43.802511 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532dd97-aa31-41d0-80be-dd06ec80e037 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:08:43.803008 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532dd97-aa31-41d0-80be-dd06ec80e037 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:08:43.803128 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532dd97-aa31-41d0-80be-dd06ec80e037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:08:43.822335 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1532dd97-aa31-41d0-80be-dd06ec80e037 None None] MFA Rules not processed for user `4cc1db8ef8c44763a0e5f62a3404a834`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:08:43.823541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1532dd97-aa31-41d0-80be-dd06ec80e037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:08:43.878179 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1401/2806] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:08:53.870984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=25 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:09.725991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:09.727400 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:09.784038 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d59cfa-789c-4e53-80be-562c59734640 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:09.787656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a903c9df-ce51-4083-9260-b2f6a4917975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:09.788252 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903c9df-ce51-4083-9260-b2f6a4917975 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:09.788531 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903c9df-ce51-4083-9260-b2f6a4917975 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:09.788832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903c9df-ce51-4083-9260-b2f6a4917975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:09.790574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a903c9df-ce51-4083-9260-b2f6a4917975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:09.875323 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903c9df-ce51-4083-9260-b2f6a4917975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:09.876779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a903c9df-ce51-4083-9260-b2f6a4917975 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:09.897644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1406/2807] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:09:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:09:15.487902 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532dd97-aa31-41d0-80be-dd06ec80e037 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:15.498282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532dd97-aa31-41d0-80be-dd06ec80e037 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:15.502197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:15.502794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:15.503071 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:15.503343 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:15.505127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:15.518817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:15.520280 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:15.531422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1402/2808] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:09:18.147882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:18.148337 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:18.148441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:18.167547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] MFA Rules not processed for user `cd0c6adf0bb34ba0b33bad9d46b47ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:09:18.168891 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:18.224666 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1407/2809] 10.222.0.33 () {58 vars in 967 bytes} [Mon Oct 3 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:09:18.246422 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:18.248483 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:18.317160 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a2743-4270-4c2c-aaca-fb4db7323e91 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:18.322178 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7701e2653e4b4ca7a842887c0526862f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:18.322784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:18.323084 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:18.323357 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:18.332230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:18.334106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:18.343342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1403/2810] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Oct 3 16:09:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 16:09:24.718686 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:24.720206 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:24.752094 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=100 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:24.787532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566c140-2059-43df-8038-c294ab2dd16b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:24.792111 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:24.792766 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:24.793051 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:24.793325 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] PATH_INFO: `/v3/users/7fe42f8d88c540418580fb8c668bf2de` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:24.795217 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fe42f8d88c540418580fb8c668bf2de)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:24.797192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:24.843183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fe42f8d88c540418580fb8c668bf2de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:24.850010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fe42f8d88c540418580fb8c668bf2de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:24.862974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fe42f8d88c540418580fb8c668bf2de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:24.869790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fe42f8d88c540418580fb8c668bf2de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:24.870437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fe42f8d88c540418580fb8c668bf2de'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:24.876619 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1408/2811] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:09:24 2022] DELETE /identity/v3/users/7fe42f8d88c540418580fb8c668bf2de => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:25.064818 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:25.078559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6fcd3-49fd-4cb3-a358-cc6e035784e6 service swift] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:25.145584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:25.146262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:25.146539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:25.146807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] PATH_INFO: `/v3/projects/a7ffe330a5d2482e961c93077d13cc5b` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:25.147919 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7ffe330a5d2482e961c93077d13cc5b)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:25.149866 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:25.194409 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7ffe330a5d2482e961c93077d13cc5b', 'request_id': 'req-1f044c35-0eb6-4882-b857-1a6fe2677aaf'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.202202 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7ffe330a5d2482e961c93077d13cc5b', 'request_id': 'req-1f044c35-0eb6-4882-b857-1a6fe2677aaf'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.207687 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7ffe330a5d2482e961c93077d13cc5b', 'request_id': 'req-1f044c35-0eb6-4882-b857-1a6fe2677aaf'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.219448 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] The token cache is being invalidate because project a7ffe330a5d2482e961c93077d13cc5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:09:25.219728 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.220353 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.221929 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1404/2812] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:09:25 2022] DELETE /identity/v3/projects/a7ffe330a5d2482e961c93077d13cc5b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:25.225907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:25.227329 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:25.240356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391f670-81cc-4b44-9764-1c94cde9b9ea admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:25.296318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:25.297031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:25.297306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:25.297574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] PATH_INFO: `/v3/users/8583f7200658447b8be5a40654042622` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:25.298806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8583f7200658447b8be5a40654042622)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:25.300870 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:25.347887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8583f7200658447b8be5a40654042622'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.349580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:25.357957 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8583f7200658447b8be5a40654042622'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.370621 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8583f7200658447b8be5a40654042622'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.378037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8583f7200658447b8be5a40654042622'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.378743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8583f7200658447b8be5a40654042622'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.384661 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1409/2813] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:09:25 2022] DELETE /identity/v3/users/8583f7200658447b8be5a40654042622 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:25.559011 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:25.561320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:25.575807 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f044c35-0eb6-4882-b857-1a6fe2677aaf admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:25.579326 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:25.636521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:25.637201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:25.638055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:25.638344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] PATH_INFO: `/v3/projects/4fc33625256646a3b8bf4158052e669a` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:25.639563 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc33625256646a3b8bf4158052e669a)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:25.641559 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:25.689230 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc33625256646a3b8bf4158052e669a', 'request_id': 'req-00ed0e35-b603-4f3e-be6e-60256d3d32d8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.699156 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc33625256646a3b8bf4158052e669a', 'request_id': 'req-00ed0e35-b603-4f3e-be6e-60256d3d32d8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.705573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc33625256646a3b8bf4158052e669a', 'request_id': 'req-00ed0e35-b603-4f3e-be6e-60256d3d32d8'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.718601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] The token cache is being invalidate because project 4fc33625256646a3b8bf4158052e669a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:09:25.718817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.719474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:25.736747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1405/2814] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:09:25 2022] DELETE /identity/v3/projects/4fc33625256646a3b8bf4158052e669a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:35.381225 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:35.714482 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:38.575935 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:38.577395 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:38.633558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fad79-5a0b-4680-8411-9a41c429d222 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:38.681340 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:38.681939 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:38.682210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:38.682501 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:38.684276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:38.746140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:38.747819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:38.757527 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1410/2815] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:09:44.883486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:44.883932 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:44.946530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:44.949042 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:44.965663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649df7-e3f0-4a5d-919c-f09e34e140fc service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:44.971620 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:44.972927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:44.972927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:44.973102 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] PATH_INFO: `/v3/users/7377a1bca391446eb36b2a3233a7d471` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:44.979719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7377a1bca391446eb36b2a3233a7d471)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:44.983068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:45.015381 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0e35-b603-4f3e-be6e-60256d3d32d8 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:45.021305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:45.023168 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:45.023555 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:45.023997 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:45.026860 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:45.046034 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:45.047787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:45.062841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1406/2816] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:09:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:09:45.065741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7377a1bca391446eb36b2a3233a7d471'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.076131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7377a1bca391446eb36b2a3233a7d471'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.086229 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:45.092685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7377a1bca391446eb36b2a3233a7d471'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.102401 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7377a1bca391446eb36b2a3233a7d471'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.103455 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7377a1bca391446eb36b2a3233a7d471'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.114172 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1411/2817] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:09:44 2022] DELETE /identity/v3/users/7377a1bca391446eb36b2a3233a7d471 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:45.293848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:45.310644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b9e99-bb24-43d7-be98-8732c4e2c6ec service placement] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:45.372535 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:45.373200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:45.373481 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:45.373756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] PATH_INFO: `/v3/projects/40732e98240b4797b941e57f6606e6c5` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:45.374894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40732e98240b4797b941e57f6606e6c5)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:45.376848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:45.423593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40732e98240b4797b941e57f6606e6c5', 'request_id': 'req-1b1e5216-cec7-4cca-8272-b03cc79f0a00'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.430810 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40732e98240b4797b941e57f6606e6c5', 'request_id': 'req-1b1e5216-cec7-4cca-8272-b03cc79f0a00'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.437351 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40732e98240b4797b941e57f6606e6c5', 'request_id': 'req-1b1e5216-cec7-4cca-8272-b03cc79f0a00'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.448449 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] The token cache is being invalidate because project 40732e98240b4797b941e57f6606e6c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:09:45.448758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.449416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.451050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1407/2818] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:09:45 2022] DELETE /identity/v3/projects/40732e98240b4797b941e57f6606e6c5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:45.455260 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:45.456682 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:45.469974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0508cb3-0178-495b-8858-e8a2c4510b09 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:45.526654 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:45.527276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:45.527647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:45.527963 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] PATH_INFO: `/v3/users/4dbcb844983a46ddbf28a1ca12e3414c` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:45.529125 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dbcb844983a46ddbf28a1ca12e3414c)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:45.531025 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:45.579319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dbcb844983a46ddbf28a1ca12e3414c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.586744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dbcb844983a46ddbf28a1ca12e3414c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.598813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dbcb844983a46ddbf28a1ca12e3414c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.605538 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dbcb844983a46ddbf28a1ca12e3414c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.606729 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dbcb844983a46ddbf28a1ca12e3414c'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.612396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1412/2819] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:09:45 2022] DELETE /identity/v3/users/4dbcb844983a46ddbf28a1ca12e3414c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:45.788697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:45.790262 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:45.806306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e5216-cec7-4cca-8272-b03cc79f0a00 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:45.826017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:45.870953 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:45.872092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:45.872367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:45.872636 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] PATH_INFO: `/v3/projects/cc473a71236e4ccf9e304f1e1e309c81` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:45.873796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc473a71236e4ccf9e304f1e1e309c81)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:45.875745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:45.921696 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc473a71236e4ccf9e304f1e1e309c81', 'request_id': 'req-6e6e87f6-a172-4c24-9560-41d62067097f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.929059 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc473a71236e4ccf9e304f1e1e309c81', 'request_id': 'req-6e6e87f6-a172-4c24-9560-41d62067097f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.935641 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc473a71236e4ccf9e304f1e1e309c81', 'request_id': 'req-6e6e87f6-a172-4c24-9560-41d62067097f'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.946683 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] The token cache is being invalidate because project cc473a71236e4ccf9e304f1e1e309c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:09:45.946969 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.947644 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:45.949177 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1408/2820] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:09:45 2022] DELETE /identity/v3/projects/cc473a71236e4ccf9e304f1e1e309c81 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:47.406450 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:47.408095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:47.494091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:47.537785 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27920cbe-755a-41c2-b39c-b544c8652ff7 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:47.628031 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:47.628770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:47.629299 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:47.629765 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:47.631887 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:47.646524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:47.648467 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:47.663293 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1413/2821] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:09:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:09:55.269620 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:55.271079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:55.333318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e87f6-a172-4c24-9560-41d62067097f admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:55.337588 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:55.338191 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:55.338463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:55.338729 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:55.341613 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:55.413663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:55.415106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:55.425542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1409/2822] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:09:55.426391 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:56.223537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:56.225181 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:56.241932 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b559a15-1307-48c9-b19f-02ca2b0138d8 service nova] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:56.246985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:56.247732 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.248012 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.248282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] PATH_INFO: `/v3/users/4cc1db8ef8c44763a0e5f62a3404a834` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.249457 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc1db8ef8c44763a0e5f62a3404a834)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:56.251378 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:56.300743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc1db8ef8c44763a0e5f62a3404a834'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.309089 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc1db8ef8c44763a0e5f62a3404a834'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.325754 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc1db8ef8c44763a0e5f62a3404a834'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.332497 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc1db8ef8c44763a0e5f62a3404a834'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.333220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-be845038-2da0-473a-99a6-35dbdf82b40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc1db8ef8c44763a0e5f62a3404a834'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.339282 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1414/2823] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:09:56 2022] DELETE /identity/v3/users/4cc1db8ef8c44763a0e5f62a3404a834 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:56.509835 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:56.521274 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad3294-fed1-4017-b06d-8b6bcc24c647 service neutron] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:56.580540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:56.581198 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.581480 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.581754 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] PATH_INFO: `/v3/projects/62932ad37df941b1840b183a858c7876` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.582906 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62932ad37df941b1840b183a858c7876)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:56.585175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:56.631768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62932ad37df941b1840b183a858c7876', 'request_id': 'req-7cdf3932-bb25-4550-9b89-ef3b543c9187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.638794 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62932ad37df941b1840b183a858c7876', 'request_id': 'req-7cdf3932-bb25-4550-9b89-ef3b543c9187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.644206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62932ad37df941b1840b183a858c7876', 'request_id': 'req-7cdf3932-bb25-4550-9b89-ef3b543c9187'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.654915 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] The token cache is being invalidate because project 62932ad37df941b1840b183a858c7876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:09:56.655196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.655812 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-7cdf3932-bb25-4550-9b89-ef3b543c9187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:09:56.657321 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1410/2824] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:09:56 2022] DELETE /identity/v3/projects/62932ad37df941b1840b183a858c7876 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:56.662243 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0fc4-7493-4471-91f3-dc78cbd3e282 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.662530 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0fc4-7493-4471-91f3-dc78cbd3e282 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.662809 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0fc4-7493-4471-91f3-dc78cbd3e282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.684175 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-071a0fc4-7493-4471-91f3-dc78cbd3e282 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:09:56.685536 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-071a0fc4-7493-4471-91f3-dc78cbd3e282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:56.732744 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:56.746996 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1415/2825] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:09:56.755066 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e82c4-187b-475b-9e39-6061e26efeb7 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.755349 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e82c4-187b-475b-9e39-6061e26efeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.755622 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e82c4-187b-475b-9e39-6061e26efeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.775164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-6c4e82c4-187b-475b-9e39-6061e26efeb7 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:09:56.776371 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-6c4e82c4-187b-475b-9e39-6061e26efeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:56.784430 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1411/2826] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:09:56.788571 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a0fc4-7493-4471-91f3-dc78cbd3e282 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:56.798960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a0fc4-7493-4471-91f3-dc78cbd3e282 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:56.803521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:56.804120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.804397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.804668 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.805356 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:09:56.805688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:56.807328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:56.820009 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1416/2827] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:09:56.825731 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e82c4-187b-475b-9e39-6061e26efeb7 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:56.835259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e82c4-187b-475b-9e39-6061e26efeb7 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:56.839947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:56.840536 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.840851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.841122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.841762 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:56.843674 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:56.880474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1412/2828] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:09:56 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:09:56.884989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:56.895150 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25b61a-f0b1-4d31-819b-53c871c9cc1a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:56.951055 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:56.951647 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.951923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.952192 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.952862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:56.954759 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:56.975997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1417/2829] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:09:56 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:09:56.980158 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:56.990737 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0078d-189b-4666-9e1b-dffb15ecee37 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:56.995494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:56.996774 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:56.997052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:56.997331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:56.998092 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:56.999715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:57.010282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1413/2830] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:09:57.015122 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:57.025210 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0798dbb-384f-4473-be27-85a3be9ce471 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:57.030983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a6ab7-2e0b-4736-b1ea-4f59d9bfbef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:57.030983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6ab7-2e0b-4736-b1ea-4f59d9bfbef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:57.031408 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6ab7-2e0b-4736-b1ea-4f59d9bfbef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:57.031684 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6ab7-2e0b-4736-b1ea-4f59d9bfbef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:57.032331 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a6ab7-2e0b-4736-b1ea-4f59d9bfbef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:57.033968 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a6ab7-2e0b-4736-b1ea-4f59d9bfbef8 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:57.045379 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1418/2831] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:09:57.050053 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:57.062474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b32989-17e0-439e-a8ef-7ed04f0aed5d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:57.067961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:57.068712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:57.069022 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:57.069324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] PATH_INFO: `/v3/projects/c12492706f844793a3ae0e374e69cd04/users/d9182ee1b4c643b9ace3680a00715662/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:57.085093 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12492706f844793a3ae0e374e69cd04, user_id=d9182ee1b4c643b9ace3680a00715662, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:57.087837 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:57.102375 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1414/2832] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:09:57 2022] PUT /identity/v3/projects/c12492706f844793a3ae0e374e69cd04/users/d9182ee1b4c643b9ace3680a00715662/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:09:57.113297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c9a40-1872-4a7c-8732-c145ddc624f0 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:57.113584 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c9a40-1872-4a7c-8732-c145ddc624f0 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:57.113851 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c9a40-1872-4a7c-8732-c145ddc624f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:57.144397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b92c9a40-1872-4a7c-8732-c145ddc624f0 None None] MFA Rules not processed for user `d9182ee1b4c643b9ace3680a00715662`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:09:57.145559 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b92c9a40-1872-4a7c-8732-c145ddc624f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:57.211653 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1419/2833] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:09:57.222533 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:09:57.224203 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:09:57.244597 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:09:57.287664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de6d38-f8c5-48c2-8e29-b2d05577da1d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:09:57.291539 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624be6fd-91fc-4ac7-8d8a-d1a0ed58d2cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:09:57.292134 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be6fd-91fc-4ac7-8d8a-d1a0ed58d2cf service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:09:57.292411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be6fd-91fc-4ac7-8d8a-d1a0ed58d2cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:09:57.292681 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624be6fd-91fc-4ac7-8d8a-d1a0ed58d2cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:09:57.302360 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624be6fd-91fc-4ac7-8d8a-d1a0ed58d2cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:09:57.303802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624be6fd-91fc-4ac7-8d8a-d1a0ed58d2cf service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:09:57.313016 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1415/2834] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:05.708548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c9a40-1872-4a7c-8732-c145ddc624f0 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:05.755460 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:05.774565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c9a40-1872-4a7c-8732-c145ddc624f0 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:05.779326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:05.780205 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:05.780491 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:05.780811 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] PATH_INFO: `/v3/users/d9182ee1b4c643b9ace3680a00715662` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:05.781978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9182ee1b4c643b9ace3680a00715662)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:05.783900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:05.830741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9182ee1b4c643b9ace3680a00715662'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:05.838198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9182ee1b4c643b9ace3680a00715662'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:05.850979 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9182ee1b4c643b9ace3680a00715662'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:05.858180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9182ee1b4c643b9ace3680a00715662'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:05.858819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9182ee1b4c643b9ace3680a00715662'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:05.864688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1420/2835] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:05 2022] DELETE /identity/v3/users/d9182ee1b4c643b9ace3680a00715662 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:05.869195 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:05.869667 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:05.869769 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:05.889269 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:05.890478 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:05.952458 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1416/2836] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:05.962462 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:05.963896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:06.022679 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ef79f-fb25-4245-956b-3fb66384c2d3 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.026418 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebd1d6-b9e7-4343-866b-f24693cb5c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.027021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd1d6-b9e7-4343-866b-f24693cb5c53 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.027352 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd1d6-b9e7-4343-866b-f24693cb5c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.027850 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd1d6-b9e7-4343-866b-f24693cb5c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.037008 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebd1d6-b9e7-4343-866b-f24693cb5c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.038439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebd1d6-b9e7-4343-866b-f24693cb5c53 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.048609 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1421/2837] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:06.321065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.322494 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:06.335204 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eea6e-e9bb-4a4a-9a43-3fcadf1cda58 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.340096 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.340894 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.341173 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.341444 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] PATH_INFO: `/v3/projects/c12492706f844793a3ae0e374e69cd04` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.342584 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12492706f844793a3ae0e374e69cd04)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.344516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.398327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12492706f844793a3ae0e374e69cd04', 'request_id': 'req-bb397145-767c-4c3d-99bb-e8aa8904fdc5'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:06.405801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12492706f844793a3ae0e374e69cd04', 'request_id': 'req-bb397145-767c-4c3d-99bb-e8aa8904fdc5'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:06.412010 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12492706f844793a3ae0e374e69cd04', 'request_id': 'req-bb397145-767c-4c3d-99bb-e8aa8904fdc5'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:06.426253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] The token cache is being invalidate because project c12492706f844793a3ae0e374e69cd04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:06.426540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:06.427176 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-bb397145-767c-4c3d-99bb-e8aa8904fdc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:06.428664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1417/2838] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:06 2022] DELETE /identity/v3/projects/c12492706f844793a3ae0e374e69cd04 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:06.434194 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c648f4f-c06c-44da-bd89-1ea795704074 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.434596 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c648f4f-c06c-44da-bd89-1ea795704074 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.434863 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c648f4f-c06c-44da-bd89-1ea795704074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.454446 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6c648f4f-c06c-44da-bd89-1ea795704074 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:06.455583 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6c648f4f-c06c-44da-bd89-1ea795704074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:06.514862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1422/2839] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:06.522653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27859bac-7d43-4a4f-9f33-60837fc95b38 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.522945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27859bac-7d43-4a4f-9f33-60837fc95b38 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.523220 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27859bac-7d43-4a4f-9f33-60837fc95b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.543304 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-27859bac-7d43-4a4f-9f33-60837fc95b38 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:06.544616 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-27859bac-7d43-4a4f-9f33-60837fc95b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:06.554363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1418/2840] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:06.559394 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c648f4f-c06c-44da-bd89-1ea795704074 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.569321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c648f4f-c06c-44da-bd89-1ea795704074 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.573901 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.574824 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.575156 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.575427 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.576095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:10:06.576394 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.578029 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.584885 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:06.591093 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1423/2841] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:10:06.595374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27859bac-7d43-4a4f-9f33-60837fc95b38 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.604778 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27859bac-7d43-4a4f-9f33-60837fc95b38 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.609972 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.610570 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.610846 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.611116 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.611752 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.613709 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.641264 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1419/2842] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:10:06 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:06.645441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.655727 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607aba9-050a-4618-b024-3c2a57ca9744 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.714024 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.715204 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.715334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.715694 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.716338 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.718302 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.739558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1424/2843] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:10:06 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 16:10:06.743663 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.753862 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94775-0a52-4fc0-ad3c-980cd63eb085 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.758464 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.759052 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.759330 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.759601 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.760241 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.762006 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.771756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1420/2844] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:06.775568 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.786269 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a708d8a-685d-4dc7-9ce4-747ddb7eb5a3 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.790854 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71187d7d-d5e6-439c-a4bf-59251081b6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.791441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187d7d-d5e6-439c-a4bf-59251081b6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.791719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187d7d-d5e6-439c-a4bf-59251081b6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.791990 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187d7d-d5e6-439c-a4bf-59251081b6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.792625 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71187d7d-d5e6-439c-a4bf-59251081b6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.795238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71187d7d-d5e6-439c-a4bf-59251081b6ad admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.806131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1425/2845] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:06.810253 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.816806 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:06.820908 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3218e7-4e3a-4d7b-bf4d-715d036d82f6 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:06.826905 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:06.827549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.827825 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.828097 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] PATH_INFO: `/v3/projects/59db6d317c4846228409634459ad1698/users/7b845d94440f434ba4d711ef275fbd82/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.844468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59db6d317c4846228409634459ad1698, user_id=7b845d94440f434ba4d711ef275fbd82, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:06.847492 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:06.860823 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1421/2846] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:06 2022] PUT /identity/v3/projects/59db6d317c4846228409634459ad1698/users/7b845d94440f434ba4d711ef275fbd82/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:06.872390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec726a9-ef29-4447-9c16-757ed1be2fb6 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:06.872672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec726a9-ef29-4447-9c16-757ed1be2fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:06.872974 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec726a9-ef29-4447-9c16-757ed1be2fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:06.902922 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-bec726a9-ef29-4447-9c16-757ed1be2fb6 None None] MFA Rules not processed for user `7b845d94440f434ba4d711ef275fbd82`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:06.904264 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-bec726a9-ef29-4447-9c16-757ed1be2fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:06.968320 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1426/2847] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:10:06.982240 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:06.984389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:07.058796 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7876e-0c00-4183-989f-80d7bac1ef37 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:07.064426 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f3f71-0829-4517-9df8-9c12429d501d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed959e4bdaf44132b05ccaec68c41172', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:07.065056 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f3f71-0829-4517-9df8-9c12429d501d service nova] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:07.065339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f3f71-0829-4517-9df8-9c12429d501d service nova] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:07.065614 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f3f71-0829-4517-9df8-9c12429d501d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:07.073386 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f3f71-0829-4517-9df8-9c12429d501d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:07.075657 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f3f71-0829-4517-9df8-9c12429d501d service nova] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:07.084414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1422/2848] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 16:10:07.285076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec726a9-ef29-4447-9c16-757ed1be2fb6 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:07.349066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec726a9-ef29-4447-9c16-757ed1be2fb6 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:07.353553 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:07.354170 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:07.354440 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:07.354714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] PATH_INFO: `/v3/users/7b845d94440f434ba4d711ef275fbd82` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:07.355819 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b845d94440f434ba4d711ef275fbd82)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:07.357763 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:07.362946 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:07.405195 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b845d94440f434ba4d711ef275fbd82'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.412266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b845d94440f434ba4d711ef275fbd82'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.424334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b845d94440f434ba4d711ef275fbd82'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.431109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b845d94440f434ba4d711ef275fbd82'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.431736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b845d94440f434ba4d711ef275fbd82'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.438420 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1427/2849] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:07 2022] DELETE /identity/v3/users/7b845d94440f434ba4d711ef275fbd82 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:07.442844 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:07.443138 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:07.443416 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:07.463215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:07.464424 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:07.523829 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1423/2850] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:07.533534 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:07.534947 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:07.594126 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2c178-027f-4a4d-946a-8f7f7ad55159 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:07.597817 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb4a84-132c-4a36-9081-fa8470b5f4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:07.598398 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb4a84-132c-4a36-9081-fa8470b5f4cb service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:07.598672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb4a84-132c-4a36-9081-fa8470b5f4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:07.598938 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb4a84-132c-4a36-9081-fa8470b5f4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:07.608118 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb4a84-132c-4a36-9081-fa8470b5f4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:07.609576 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb4a84-132c-4a36-9081-fa8470b5f4cb service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:07.618847 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1428/2851] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:07.887245 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:07.889005 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:07.904047 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd328d18-b034-4bac-9695-879342e62e90 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:07.908839 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:07.909474 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:07.909875 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:07.910017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] PATH_INFO: `/v3/projects/59db6d317c4846228409634459ad1698` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:07.911244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59db6d317c4846228409634459ad1698)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:07.913125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:07.960700 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59db6d317c4846228409634459ad1698', 'request_id': 'req-f914384b-91a4-4670-99ed-529fdf323d96'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.968541 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59db6d317c4846228409634459ad1698', 'request_id': 'req-f914384b-91a4-4670-99ed-529fdf323d96'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.973040 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:07.975595 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59db6d317c4846228409634459ad1698', 'request_id': 'req-f914384b-91a4-4670-99ed-529fdf323d96'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.988486 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] The token cache is being invalidate because project 59db6d317c4846228409634459ad1698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:07.989112 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.989439 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-f914384b-91a4-4670-99ed-529fdf323d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:07.990923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1424/2852] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:07 2022] DELETE /identity/v3/projects/59db6d317c4846228409634459ad1698 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:07.996010 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9980-cb8d-4c61-b4b6-330b5bfee82b None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:07.997405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9980-cb8d-4c61-b4b6-330b5bfee82b None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:07.997405 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc9980-cb8d-4c61-b4b6-330b5bfee82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.016816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4ebc9980-cb8d-4c61-b4b6-330b5bfee82b None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:08.018021 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4ebc9980-cb8d-4c61-b4b6-330b5bfee82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:08.078117 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1429/2853] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:08.086196 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ac420-29fc-4eae-91ae-c9488a9753c4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.086514 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ac420-29fc-4eae-91ae-c9488a9753c4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.086793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ac420-29fc-4eae-91ae-c9488a9753c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.106841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-295ac420-29fc-4eae-91ae-c9488a9753c4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:08.108050 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-295ac420-29fc-4eae-91ae-c9488a9753c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:08.115984 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1425/2854] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:08.120047 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc9980-cb8d-4c61-b4b6-330b5bfee82b None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.130286 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc9980-cb8d-4c61-b4b6-330b5bfee82b None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.134816 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.135439 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.135715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.135985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.136705 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:10:08.137068 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.138672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.150128 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1430/2855] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:10:08.154890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ac420-29fc-4eae-91ae-c9488a9753c4 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.164159 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ac420-29fc-4eae-91ae-c9488a9753c4 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.169447 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.170065 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.170346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.170619 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.171364 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.173329 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.200024 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1426/2856] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:10:08 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:08.204148 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.214218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522591e1-9749-4319-8db3-3cc27953752a admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.225923 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:08.272799 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.273399 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.273672 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.273941 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.274567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.276473 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.297735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1431/2857] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:10:08 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:08.302013 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.313721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18396666-943d-4d95-8808-f5d72abc77ba admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.318652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.319153 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.319429 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.319699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.320339 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.321992 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.332799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1427/2858] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:08.337064 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.347948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196e1c-2b0b-4c58-b320-3e21cbce8859 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.352560 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a693e6-dd5f-40ba-87a6-bbdc5062e8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.353183 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a693e6-dd5f-40ba-87a6-bbdc5062e8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.353463 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a693e6-dd5f-40ba-87a6-bbdc5062e8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.353734 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a693e6-dd5f-40ba-87a6-bbdc5062e8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.354368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a693e6-dd5f-40ba-87a6-bbdc5062e8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.357189 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a693e6-dd5f-40ba-87a6-bbdc5062e8f3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.366888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1432/2859] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:08.370788 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.381218 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce646c6-a680-464f-916d-771081c13a28 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.385838 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.386469 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.387163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.387163 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] PATH_INFO: `/v3/projects/4d9d9806ef2a46f18d3133b777b6c279/users/9acd48a336ad41cebb424e1bcc9b8b80/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.401332 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9d9806ef2a46f18d3133b777b6c279, user_id=9acd48a336ad41cebb424e1bcc9b8b80, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.404787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.418698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1428/2860] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:08 2022] PUT /identity/v3/projects/4d9d9806ef2a46f18d3133b777b6c279/users/9acd48a336ad41cebb424e1bcc9b8b80/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:08.429076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32d9b4-1f2f-41c8-81eb-7067b6209bb8 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.429364 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32d9b4-1f2f-41c8-81eb-7067b6209bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.429638 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32d9b4-1f2f-41c8-81eb-7067b6209bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.457533 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-eb32d9b4-1f2f-41c8-81eb-7067b6209bb8 None None] MFA Rules not processed for user `9acd48a336ad41cebb424e1bcc9b8b80`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:08.458772 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-eb32d9b4-1f2f-41c8-81eb-7067b6209bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:08.522657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1433/2861] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:08.527148 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.590374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed20b-9c2d-4358-b19c-a26195efea15 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.595604 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.596199 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.596477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.596775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.597435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.599363 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.626324 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1429/2862] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:10:08 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:08.630382 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32d9b4-1f2f-41c8-81eb-7067b6209bb8 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.640888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32d9b4-1f2f-41c8-81eb-7067b6209bb8 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.695485 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.696075 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.696347 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.696615 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.697255 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.699159 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.720548 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1434/2863] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:10:08 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:08.724669 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.735485 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6907ec-2f5a-4531-8281-887d1f5d4164 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.740106 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.740692 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.740996 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.741267 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.741899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.743506 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.754058 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1430/2864] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:08.759690 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.769820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031979f-e965-48cc-8cba-4394b0fde319 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.774423 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.775098 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.775375 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.775644 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] PATH_INFO: `/v3/projects/c8ac2a032fbb437daaedb83efbe2bfd9/users/529245b28e904fa1822caf58b98a747d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.787413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:08.791805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ac2a032fbb437daaedb83efbe2bfd9, user_id=529245b28e904fa1822caf58b98a747d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.795437 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.808831 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1435/2865] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:08 2022] PUT /identity/v3/projects/c8ac2a032fbb437daaedb83efbe2bfd9/users/529245b28e904fa1822caf58b98a747d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:08.812897 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.822948 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77389d-808d-4ba9-91eb-65e7208531f7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.880573 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8852e9-a4a5-487f-bbcd-2f8bb6ebefb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.881197 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8852e9-a4a5-487f-bbcd-2f8bb6ebefb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.881477 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8852e9-a4a5-487f-bbcd-2f8bb6ebefb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.881749 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8852e9-a4a5-487f-bbcd-2f8bb6ebefb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.882388 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8852e9-a4a5-487f-bbcd-2f8bb6ebefb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.883991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8852e9-a4a5-487f-bbcd-2f8bb6ebefb0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.885242 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:08.895412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1431/2866] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:08.899582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:08.910276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7442252-7aec-42f7-b17e-c93429b2d327 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:08.915909 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:08.916551 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.916869 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.917129 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] PATH_INFO: `/v3/projects/c8ac2a032fbb437daaedb83efbe2bfd9/users/529245b28e904fa1822caf58b98a747d/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:08.919057 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ac2a032fbb437daaedb83efbe2bfd9, user_id=529245b28e904fa1822caf58b98a747d, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:08.921873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:08.931919 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1436/2867] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:08 2022] PUT /identity/v3/projects/c8ac2a032fbb437daaedb83efbe2bfd9/users/529245b28e904fa1822caf58b98a747d/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:08.995305 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cc115-6115-431a-a326-2fa57da855c5 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:08.995817 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cc115-6115-431a-a326-2fa57da855c5 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:08.995925 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cc115-6115-431a-a326-2fa57da855c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:09.024991 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-eb0cc115-6115-431a-a326-2fa57da855c5 None None] MFA Rules not processed for user `529245b28e904fa1822caf58b98a747d`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:09.026187 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-eb0cc115-6115-431a-a326-2fa57da855c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:09.093768 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1432/2868] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:09.105037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:09.106803 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:09.163480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006b7af-69bf-412f-bcd8-b91637707337 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:09.167016 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:09.167608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:09.167883 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:09.168149 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:09.176948 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:09.178368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:09.187574 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1437/2869] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:09.274516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cc115-6115-431a-a326-2fa57da855c5 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:09.285082 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cc115-6115-431a-a326-2fa57da855c5 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:09.288671 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b8361-2bb9-43ed-bc3e-58d12399085b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:09.289288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b8361-2bb9-43ed-bc3e-58d12399085b service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:09.289566 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b8361-2bb9-43ed-bc3e-58d12399085b service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:09.289833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b8361-2bb9-43ed-bc3e-58d12399085b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:09.351851 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b8361-2bb9-43ed-bc3e-58d12399085b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:09.353365 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b8361-2bb9-43ed-bc3e-58d12399085b service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:09.362206 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1433/2870] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:18.513752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:18.575344 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f6a33-80c9-4d98-af08-c4c6b1e73907 service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:18.580107 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:18.580736 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:18.581032 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:18.581301 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] PATH_INFO: `/v3/users/9acd48a336ad41cebb424e1bcc9b8b80` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:18.582413 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acd48a336ad41cebb424e1bcc9b8b80)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:18.584322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:18.630200 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acd48a336ad41cebb424e1bcc9b8b80'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:18.637524 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acd48a336ad41cebb424e1bcc9b8b80'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:18.649547 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acd48a336ad41cebb424e1bcc9b8b80'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:18.656832 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acd48a336ad41cebb424e1bcc9b8b80'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:18.657459 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acd48a336ad41cebb424e1bcc9b8b80'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:18.658834 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:18.665132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1438/2871] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:18 2022] DELETE /identity/v3/users/9acd48a336ad41cebb424e1bcc9b8b80 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:18.670393 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:18.670874 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:18.670994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:18.693137 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:18.694346 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:18.755880 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1434/2872] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:18.766477 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:18.767897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:18.827271 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b98d8a-668a-4944-b6e3-4869d7aafad6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:18.830978 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:18.831582 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:18.831862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:18.832132 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:18.842006 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:18.843464 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:18.852862 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1439/2873] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:19.123175 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:19.124654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:19.136632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dae055-2ee8-46c0-a5b9-6fdad0ed517f None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:19.143446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:19.144278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:19.144549 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:19.144852 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] PATH_INFO: `/v3/projects/4d9d9806ef2a46f18d3133b777b6c279` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:19.145982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9d9806ef2a46f18d3133b777b6c279)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:19.148457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:19.167086 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=100 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:19.209389 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9d9806ef2a46f18d3133b777b6c279', 'request_id': 'req-50aad5d4-1e28-49dd-aa6f-27a1e024f408'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.217888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9d9806ef2a46f18d3133b777b6c279', 'request_id': 'req-50aad5d4-1e28-49dd-aa6f-27a1e024f408'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.223705 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9d9806ef2a46f18d3133b777b6c279', 'request_id': 'req-50aad5d4-1e28-49dd-aa6f-27a1e024f408'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.236126 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] The token cache is being invalidate because project 4d9d9806ef2a46f18d3133b777b6c279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:19.236423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.237111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.238602 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1435/2874] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:19 2022] DELETE /identity/v3/projects/4d9d9806ef2a46f18d3133b777b6c279 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:19.242688 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:19.252590 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18828737-a24d-4824-ab20-a0b5284f3a0e service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:19.310900 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:19.311652 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:19.312087 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:19.312474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] PATH_INFO: `/v3/users/529245b28e904fa1822caf58b98a747d` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:19.315432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=529245b28e904fa1822caf58b98a747d)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:19.318545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:19.368805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529245b28e904fa1822caf58b98a747d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.377577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529245b28e904fa1822caf58b98a747d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.391513 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529245b28e904fa1822caf58b98a747d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.398746 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529245b28e904fa1822caf58b98a747d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.399474 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-241b2fed-7986-4ef0-9673-8f970a8bb6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529245b28e904fa1822caf58b98a747d'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.405716 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1440/2875] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:19 2022] DELETE /identity/v3/users/529245b28e904fa1822caf58b98a747d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:19.675309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:19.676864 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:19.692215 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aad5d4-1e28-49dd-aa6f-27a1e024f408 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:19.750282 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:19.750914 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:19.751193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:19.751465 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] PATH_INFO: `/v3/projects/c8ac2a032fbb437daaedb83efbe2bfd9` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:19.752586 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ac2a032fbb437daaedb83efbe2bfd9)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:19.754579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:19.800920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ac2a032fbb437daaedb83efbe2bfd9', 'request_id': 'req-874b4def-c654-4d1a-be1b-07cfeeca12de'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.808110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ac2a032fbb437daaedb83efbe2bfd9', 'request_id': 'req-874b4def-c654-4d1a-be1b-07cfeeca12de'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.814017 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ac2a032fbb437daaedb83efbe2bfd9', 'request_id': 'req-874b4def-c654-4d1a-be1b-07cfeeca12de'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.826716 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] The token cache is being invalidate because project c8ac2a032fbb437daaedb83efbe2bfd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:19.826994 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.827745 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-874b4def-c654-4d1a-be1b-07cfeeca12de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:19.829463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1436/2876] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:19 2022] DELETE /identity/v3/projects/c8ac2a032fbb437daaedb83efbe2bfd9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:19.834578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647c5f-0dd5-4d3e-9151-40cbe62cccea None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:19.835326 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647c5f-0dd5-4d3e-9151-40cbe62cccea None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:19.835436 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647c5f-0dd5-4d3e-9151-40cbe62cccea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:19.856084 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-b5647c5f-0dd5-4d3e-9151-40cbe62cccea None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:19.858127 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-b5647c5f-0dd5-4d3e-9151-40cbe62cccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:19.870603 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:19.916871 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1441/2877] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:19.925518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f306a-745d-4949-9c96-8db1c8dac495 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:19.925804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f306a-745d-4949-9c96-8db1c8dac495 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:19.926079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f306a-745d-4949-9c96-8db1c8dac495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:19.946771 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-1b9f306a-745d-4949-9c96-8db1c8dac495 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:19.947982 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-1b9f306a-745d-4949-9c96-8db1c8dac495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:19.956089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1437/2878] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 16:10:19.960980 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5647c5f-0dd5-4d3e-9151-40cbe62cccea None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:19.971290 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5647c5f-0dd5-4d3e-9151-40cbe62cccea None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:19.975945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:19.976546 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:19.976864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:19.977140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:19.977813 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:10:19.978114 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:19.979712 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:19.991663 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1442/2879] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:10:19.995900 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f306a-745d-4949-9c96-8db1c8dac495 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.005459 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f306a-745d-4949-9c96-8db1c8dac495 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.010125 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.010712 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.010988 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.011261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.011888 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.014033 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.043278 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1438/2880] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:10:19 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:20.044798 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.055741 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb8691-01ec-4bd8-8094-ea84f20c3af1 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.111983 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.112580 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.112888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.113155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.113779 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.115685 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.135806 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1443/2881] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:10:20 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:20.140396 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.147617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:20.151661 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd66bd4-6b88-4001-863a-d35d1150c6c7 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.158500 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.159300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.159747 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.160038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.160756 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.162556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.174080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1439/2882] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:20.178249 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.188612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958212-04d5-473f-ae03-d7274591d2fa admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.193297 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f371f8-6f9a-408f-aec3-82d9443ee918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.193896 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f371f8-6f9a-408f-aec3-82d9443ee918 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.194176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f371f8-6f9a-408f-aec3-82d9443ee918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.194447 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f371f8-6f9a-408f-aec3-82d9443ee918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.195092 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f371f8-6f9a-408f-aec3-82d9443ee918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.196691 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f371f8-6f9a-408f-aec3-82d9443ee918 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.208367 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1444/2883] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:20.212827 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.224291 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b871674-33a9-43c9-9867-9246f4c75caa admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.228924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.229571 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.229847 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.230136 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] PATH_INFO: `/v3/projects/bcc44548ff224994911697a077ce56fd/users/ba0e01a72b19489ea7254f24313dbdfb/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.246926 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc44548ff224994911697a077ce56fd, user_id=ba0e01a72b19489ea7254f24313dbdfb, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.250232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.264653 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1440/2884] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:20 2022] PUT /identity/v3/projects/bcc44548ff224994911697a077ce56fd/users/ba0e01a72b19489ea7254f24313dbdfb/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:20.276109 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf642e9-6565-41e2-8699-e8e9111e8c16 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.276389 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf642e9-6565-41e2-8699-e8e9111e8c16 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.276657 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf642e9-6565-41e2-8699-e8e9111e8c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.304907 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-8bf642e9-6565-41e2-8699-e8e9111e8c16 None None] MFA Rules not processed for user `ba0e01a72b19489ea7254f24313dbdfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:20.306099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-8bf642e9-6565-41e2-8699-e8e9111e8c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:20.371396 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1445/2885] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:20.375945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.439244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0690e-1373-4748-8337-725f8e3ac3af admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.443899 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.444522 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.444833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.445107 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.445734 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.447772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.473576 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1441/2886] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:10:20 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:20.477760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf642e9-6565-41e2-8699-e8e9111e8c16 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.488738 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf642e9-6565-41e2-8699-e8e9111e8c16 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.500537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:20.546432 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.547040 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.547318 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.547589 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.548218 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.550176 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.571321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1446/2887] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:10:20 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:20.575456 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.586110 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98b29c-0244-42ff-bb35-cfc2f07878b4 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.590698 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.591890 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.592167 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.592435 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.593111 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.594715 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.605122 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1442/2888] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:20.610027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.620155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e0b94-2843-456f-be3f-ed429c3f20c3 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.625023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.625715 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.625999 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.626276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] PATH_INFO: `/v3/projects/843aa6720e834d799db3727019d526c9/users/6742949e0d23468aab69709f6e600ae6/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.642366 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843aa6720e834d799db3727019d526c9, user_id=6742949e0d23468aab69709f6e600ae6, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.645066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.657558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1447/2889] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:20 2022] PUT /identity/v3/projects/843aa6720e834d799db3727019d526c9/users/6742949e0d23468aab69709f6e600ae6/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:20.661540 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.671947 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd98d2-6aa8-4613-8c9d-5eeb4e949de5 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.727117 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd6586-94f7-4390-8f51-a5e49e418344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.727748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6586-94f7-4390-8f51-a5e49e418344 admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.728001 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6586-94f7-4390-8f51-a5e49e418344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.728309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6586-94f7-4390-8f51-a5e49e418344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.729037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd6586-94f7-4390-8f51-a5e49e418344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.730634 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd6586-94f7-4390-8f51-a5e49e418344 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.740822 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1443/2890] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:10:20.744921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.755532 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3a6af-9e95-4ca1-ae78-ea11dc3b4c45 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.760144 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.760842 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.761124 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.761397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] PATH_INFO: `/v3/projects/843aa6720e834d799db3727019d526c9/users/6742949e0d23468aab69709f6e600ae6/roles/0cb19d985d3d456983e0b1624610b058` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.763332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843aa6720e834d799db3727019d526c9, user_id=6742949e0d23468aab69709f6e600ae6, role_id=0cb19d985d3d456983e0b1624610b058)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.766279 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.776276 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1448/2891] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:20 2022] PUT /identity/v3/projects/843aa6720e834d799db3727019d526c9/users/6742949e0d23468aab69709f6e600ae6/roles/0cb19d985d3d456983e0b1624610b058 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:10:20.786961 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.787244 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.787515 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.815918 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] MFA Rules not processed for user `6742949e0d23468aab69709f6e600ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:20.817151 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:20.845632 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:20.884801 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1444/2892] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:20.896027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:20.897820 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:20.955829 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173520da-9003-4eee-b10a-9ffaca128fc8 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:20.959403 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:20.960266 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:20.960529 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:20.960827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:20.970020 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:20.971444 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:20.980770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1449/2893] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:21.028412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:21.030030 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:21.041980 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00597158-9767-4a7c-8a8b-d669bca5e3f8 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:21.046318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583c71c-2047-48ff-af88-be14f477716d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:21.046916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583c71c-2047-48ff-af88-be14f477716d service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:21.047193 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583c71c-2047-48ff-af88-be14f477716d service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:21.047461 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583c71c-2047-48ff-af88-be14f477716d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:21.107989 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583c71c-2047-48ff-af88-be14f477716d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:21.109495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583c71c-2047-48ff-af88-be14f477716d service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:21.118664 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1445/2894] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:24.416519 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:24.477786 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaf820-40ab-457c-a5c6-5779cd4ff53f service cinder] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:24.482770 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:24.483381 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:24.483651 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:24.483916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] PATH_INFO: `/v3/users/ba0e01a72b19489ea7254f24313dbdfb` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:24.485049 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0e01a72b19489ea7254f24313dbdfb)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:24.486945 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:24.499065 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:24.533088 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0e01a72b19489ea7254f24313dbdfb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:24.539921 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0e01a72b19489ea7254f24313dbdfb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:24.552465 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0e01a72b19489ea7254f24313dbdfb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:24.559253 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0e01a72b19489ea7254f24313dbdfb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:24.559916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0e01a72b19489ea7254f24313dbdfb'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:24.565714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1450/2895] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:24 2022] DELETE /identity/v3/users/ba0e01a72b19489ea7254f24313dbdfb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:24.570263 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:24.570743 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:24.570950 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:24.590359 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:24.592519 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:24.650670 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1446/2896] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:24.660545 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:24.661972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:24.722989 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fe68d-b8c3-47ef-b6c0-63c2a0efd3ff admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:24.726595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:24.727208 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:24.727577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:24.727774 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:24.737594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:24.739045 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:24.748960 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1451/2897] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:25.011802 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:25.013484 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:25.026693 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd982a-d715-4e79-8fba-5e6a9fc096d0 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:25.033038 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:25.034043 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:25.034337 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:25.034799 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] PATH_INFO: `/v3/projects/bcc44548ff224994911697a077ce56fd` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:25.036062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc44548ff224994911697a077ce56fd)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:25.038032 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:25.088868 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc44548ff224994911697a077ce56fd', 'request_id': 'req-9294d63e-488e-4903-8eea-aed9be4fbfb0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.098295 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc44548ff224994911697a077ce56fd', 'request_id': 'req-9294d63e-488e-4903-8eea-aed9be4fbfb0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.100518 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:25.105362 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc44548ff224994911697a077ce56fd', 'request_id': 'req-9294d63e-488e-4903-8eea-aed9be4fbfb0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.118127 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] The token cache is being invalidate because project bcc44548ff224994911697a077ce56fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:25.118417 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.119079 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.120575 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1447/2898] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:25 2022] DELETE /identity/v3/projects/bcc44548ff224994911697a077ce56fd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:25.124612 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:25.134306 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cb575-41eb-4b02-b084-54f65a648db5 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:25.195289 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:25.196359 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:25.196795 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:25.197069 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] PATH_INFO: `/v3/users/6742949e0d23468aab69709f6e600ae6` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:25.198198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6742949e0d23468aab69709f6e600ae6)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:25.201044 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:25.238717 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:25.249565 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6742949e0d23468aab69709f6e600ae6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.257056 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6742949e0d23468aab69709f6e600ae6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.269927 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6742949e0d23468aab69709f6e600ae6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.276552 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6742949e0d23468aab69709f6e600ae6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.277233 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-27878ec2-3044-4591-871c-df283cd17c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6742949e0d23468aab69709f6e600ae6'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.283577 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1452/2899] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:25 2022] DELETE /identity/v3/users/6742949e0d23468aab69709f6e600ae6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:25.559758 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:25.561318 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:25.579910 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294d63e-488e-4903-8eea-aed9be4fbfb0 admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:25.637232 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:25.637785 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:25.638593 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:25.638957 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] PATH_INFO: `/v3/projects/843aa6720e834d799db3727019d526c9` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:25.640078 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=843aa6720e834d799db3727019d526c9)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:25.642019 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:25.686579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843aa6720e834d799db3727019d526c9', 'request_id': 'req-26942b36-6049-4887-b806-f1efafad2fa9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.693695 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843aa6720e834d799db3727019d526c9', 'request_id': 'req-26942b36-6049-4887-b806-f1efafad2fa9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.699528 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843aa6720e834d799db3727019d526c9', 'request_id': 'req-26942b36-6049-4887-b806-f1efafad2fa9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.711049 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] The token cache is being invalidate because project 843aa6720e834d799db3727019d526c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:25.711331 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.711970 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-26942b36-6049-4887-b806-f1efafad2fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:25.713457 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1448/2900] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:25 2022] DELETE /identity/v3/projects/843aa6720e834d799db3727019d526c9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:25.718997 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82923427-275e-450b-b162-2788a9b016c5 None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:25.719496 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82923427-275e-450b-b162-2788a9b016c5 None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:25.719629 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82923427-275e-450b-b162-2788a9b016c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:25.739263 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-82923427-275e-450b-b162-2788a9b016c5 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:25.740451 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-82923427-275e-450b-b162-2788a9b016c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:25.798368 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1453/2901] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:25.806387 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979269b3-ea4d-43d5-bbf1-21e25054d040 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:25.806652 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979269b3-ea4d-43d5-bbf1-21e25054d040 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:25.806923 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979269b3-ea4d-43d5-bbf1-21e25054d040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:25.828473 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-979269b3-ea4d-43d5-bbf1-21e25054d040 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:25.829706 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-979269b3-ea4d-43d5-bbf1-21e25054d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:25.839516 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1449/2902] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:25.843931 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82923427-275e-450b-b162-2788a9b016c5 None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:25.858953 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82923427-275e-450b-b162-2788a9b016c5 None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:25.863743 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:25.864353 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:25.864633 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:25.864929 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:25.865578 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:10:25.865882 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:25.867486 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:25.879720 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1454/2903] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:10:25.884288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979269b3-ea4d-43d5-bbf1-21e25054d040 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:25.894313 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979269b3-ea4d-43d5-bbf1-21e25054d040 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:25.898920 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:25.899505 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:25.899780 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:25.900044 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:25.900673 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:25.902589 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:25.929954 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1450/2904] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:10:25 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:25.933881 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:25.944441 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96717a5-3071-4a47-ab79-6e241f67b192 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:26.000487 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:26.001155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:26.001428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:26.001693 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:26.002322 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:26.004254 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:26.024918 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1455/2905] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:10:25 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:26.029487 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:26.039833 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b336-a86f-4497-8395-d58d085b4d7a admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:26.045328 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:26.045927 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:26.046201 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:26.046468 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:26.047103 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:26.048742 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:26.058748 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1451/2906] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:26.062944 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:26.072760 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8a828-328e-489f-b9e0-66db29d3d8e6 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:26.077248 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c6f81-6728-4e44-8fe9-604bf612c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:26.077857 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c6f81-6728-4e44-8fe9-604bf612c7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:26.078131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c6f81-6728-4e44-8fe9-604bf612c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:26.078397 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c6f81-6728-4e44-8fe9-604bf612c7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:26.079033 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c6f81-6728-4e44-8fe9-604bf612c7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:26.080608 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c6f81-6728-4e44-8fe9-604bf612c7fc admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:26.091737 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1456/2907] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:26.096260 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:26.106098 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffeec1e-98bd-4ed7-b9c0-2c60794ce40c admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:26.111200 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:26.111848 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:26.112102 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:26.112367 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] PATH_INFO: `/v3/projects/5186bbf05dfb497583a0e7251047e2fd/users/f3f06c5a57e24069a1837cd7c290d476/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:26.126629 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5186bbf05dfb497583a0e7251047e2fd, user_id=f3f06c5a57e24069a1837cd7c290d476, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:26.129327 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:26.142699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1452/2908] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:26 2022] PUT /identity/v3/projects/5186bbf05dfb497583a0e7251047e2fd/users/f3f06c5a57e24069a1837cd7c290d476/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:26.154914 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e873dc-f617-4309-a8f8-ecdb4cccd27a None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:26.155155 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e873dc-f617-4309-a8f8-ecdb4cccd27a None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:26.155428 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e873dc-f617-4309-a8f8-ecdb4cccd27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:26.183154 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-e3e873dc-f617-4309-a8f8-ecdb4cccd27a None None] MFA Rules not processed for user `f3f06c5a57e24069a1837cd7c290d476`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:26.184351 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-e3e873dc-f617-4309-a8f8-ecdb4cccd27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:26.189521 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:26.250665 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1457/2909] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:26.261689 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:26.264037 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:26.279259 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:26.324311 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dfd9a-7408-471d-88a7-6e95a7eec65d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:26.328916 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b6abe-e56f-4c54-b858-fd209b6ed73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:26.329520 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6abe-e56f-4c54-b858-fd209b6ed73a service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:26.329798 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6abe-e56f-4c54-b858-fd209b6ed73a service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:26.330069 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b6abe-e56f-4c54-b858-fd209b6ed73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:26.338773 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b6abe-e56f-4c54-b858-fd209b6ed73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:26.340479 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b6abe-e56f-4c54-b858-fd209b6ed73a service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:26.350320 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1453/2910] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:36.247640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=24 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:36.345944 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=20 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:44.997083 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e873dc-f617-4309-a8f8-ecdb4cccd27a None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:45.060334 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e873dc-f617-4309-a8f8-ecdb4cccd27a None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:45.064910 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:45.065537 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.065808 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.066076 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] PATH_INFO: `/v3/users/f3f06c5a57e24069a1837cd7c290d476` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.067238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f06c5a57e24069a1837cd7c290d476)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:45.069180 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:45.116321 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f06c5a57e24069a1837cd7c290d476'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.123740 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f06c5a57e24069a1837cd7c290d476'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.136007 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f06c5a57e24069a1837cd7c290d476'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.143656 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f06c5a57e24069a1837cd7c290d476'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.144316 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f06c5a57e24069a1837cd7c290d476'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.150278 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1458/2911] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:44 2022] DELETE /identity/v3/users/f3f06c5a57e24069a1837cd7c290d476 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:45.154772 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.155300 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.155411 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.175441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:45.176626 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:45.234999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1454/2912] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:45.244872 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:45.246324 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:45.304913 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27afe2-f040-49e2-b08e-3a0ebc8f79dd admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:45.308586 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c98266-ca65-4dae-8300-346470d71a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:45.309220 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c98266-ca65-4dae-8300-346470d71a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.309498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c98266-ca65-4dae-8300-346470d71a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.309769 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c98266-ca65-4dae-8300-346470d71a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.319735 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c98266-ca65-4dae-8300-346470d71a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:45.321165 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c98266-ca65-4dae-8300-346470d71a39 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:45.331494 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1459/2913] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:45.600624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:45.602171 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:45.614446 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75240653-8385-464d-8813-cdc8d11e3131 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:45.619556 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:45.620314 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.621585 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.622226 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] PATH_INFO: `/v3/projects/5186bbf05dfb497583a0e7251047e2fd` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.623495 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5186bbf05dfb497583a0e7251047e2fd)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:45.626306 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:45.676036 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5186bbf05dfb497583a0e7251047e2fd', 'request_id': 'req-2076fc26-7905-4e11-8b92-5f496e5d7cd0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.683247 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5186bbf05dfb497583a0e7251047e2fd', 'request_id': 'req-2076fc26-7905-4e11-8b92-5f496e5d7cd0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.689227 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5186bbf05dfb497583a0e7251047e2fd', 'request_id': 'req-2076fc26-7905-4e11-8b92-5f496e5d7cd0'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.701025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] The token cache is being invalidate because project 5186bbf05dfb497583a0e7251047e2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:45.701025 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.701699 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-2076fc26-7905-4e11-8b92-5f496e5d7cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:45.704139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1455/2914] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:45 2022] DELETE /identity/v3/projects/5186bbf05dfb497583a0e7251047e2fd => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:45.709023 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2e59-9679-4802-8173-7f9e92009fdf None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.709313 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2e59-9679-4802-8173-7f9e92009fdf None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.709595 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2e59-9679-4802-8173-7f9e92009fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.731140 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-6b8f2e59-9679-4802-8173-7f9e92009fdf None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:45.732343 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-6b8f2e59-9679-4802-8173-7f9e92009fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:45.749438 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:45.795762 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1460/2915] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:45.804133 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1e7a-3b55-412a-afd6-2cea0bbea0e4 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.804412 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1e7a-3b55-412a-afd6-2cea0bbea0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.804686 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da1e7a-3b55-412a-afd6-2cea0bbea0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.825423 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-c5da1e7a-3b55-412a-afd6-2cea0bbea0e4 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:45.826624 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-c5da1e7a-3b55-412a-afd6-2cea0bbea0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:45.835394 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1456/2916] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:45.839635 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2e59-9679-4802-8173-7f9e92009fdf None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:45.849641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2e59-9679-4802-8173-7f9e92009fdf None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:45.854198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:45.854790 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.855066 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.855332 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] PATH_INFO: `/v3/domains` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.855966 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81368) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 16:10:45.856265 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:45.857873 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:45.870120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1461/2917] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Oct 3 16:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 16:10:45.874288 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1e7a-3b55-412a-afd6-2cea0bbea0e4 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:45.883983 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da1e7a-3b55-412a-afd6-2cea0bbea0e4 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:45.888639 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:45.889250 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.889521 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.889787 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.890463 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:45.892414 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:45.919080 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1457/2918] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Oct 3 16:10:45 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:45.924037 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:45.934048 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9f295-2c00-4d69-9ec1-beb81cbb07be admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:45.989640 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:45.990230 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:45.990504 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:45.990771 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] PATH_INFO: `/v3/users` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:45.991402 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:45.993328 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:46.014053 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1462/2919] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Oct 3 16:10:45 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 16:10:46.018433 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:46.028647 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a34a5-2f71-4ced-9c31-df9a9577ab7d admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:46.034139 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:46.034721 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:46.034995 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:46.035261 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:46.035924 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:46.037841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:46.048579 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1458/2920] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 16:10:46.052454 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:46.063288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a871c-1aea-4000-b104-401e5448a508 admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:46.067916 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4aa36-9eec-4a0c-b428-e924e08ba69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:46.068500 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4aa36-9eec-4a0c-b428-e924e08ba69a admin admin] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:46.068805 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4aa36-9eec-4a0c-b428-e924e08ba69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:46.069095 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4aa36-9eec-4a0c-b428-e924e08ba69a admin admin] PATH_INFO: `/v3/roles` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:46.069707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4aa36-9eec-4a0c-b428-e924e08ba69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:46.071291 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4aa36-9eec-4a0c-b428-e924e08ba69a admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:46.081752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1463/2921] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Oct 3 16:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 16:10:46.086027 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:46.096404 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f2ce3-601b-43da-b303-6909d1a81cde admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:46.101380 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:46.102062 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] REQUEST_METHOD: `PUT` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:46.102342 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:46.102617 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] PATH_INFO: `/v3/projects/cfb36a6004e44735b8a085010b3ff7ab/users/1b683dd6f70349d5aa3cd10d59ee94ad/roles/5a57b9470d8a4718a12b9e65777a95df` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:46.118886 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb36a6004e44735b8a085010b3ff7ab, user_id=1b683dd6f70349d5aa3cd10d59ee94ad, role_id=5a57b9470d8a4718a12b9e65777a95df)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:46.121654 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:46.134804 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1459/2922] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Oct 3 16:10:46 2022] PUT /identity/v3/projects/cfb36a6004e44735b8a085010b3ff7ab/users/1b683dd6f70349d5aa3cd10d59ee94ad/roles/5a57b9470d8a4718a12b9e65777a95df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 16:10:46.146692 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fe380-f814-4509-8daf-a3ee83fc757e None None] REQUEST_METHOD: `POST` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:46.146972 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fe380-f814-4509-8daf-a3ee83fc757e None None] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:46.147238 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fe380-f814-4509-8daf-a3ee83fc757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:46.176251 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.auth.core [None req-4d0fe380-f814-4509-8daf-a3ee83fc757e None None] MFA Rules not processed for user `1b683dd6f70349d5aa3cd10d59ee94ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81368) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:46.177498 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-4d0fe380-f814-4509-8daf-a3ee83fc757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:46.240593 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1464/2923] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:46.251537 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:46.253281 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:46.274704 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:46.317472 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53355b88-a0c3-430c-9fb1-29c583d4e10b admin admin] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:46.321174 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da27ab3-a8f8-4afa-b313-aa59f2435765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4396302927da47a3a80eae4c50fdb266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:46.321775 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da27ab3-a8f8-4afa-b313-aa59f2435765 service cinder] REQUEST_METHOD: `GET` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:46.322048 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da27ab3-a8f8-4afa-b313-aa59f2435765 service cinder] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:46.322312 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da27ab3-a8f8-4afa-b313-aa59f2435765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:46.330784 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da27ab3-a8f8-4afa-b313-aa59f2435765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:46.332212 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da27ab3-a8f8-4afa-b313-aa59f2435765 service cinder] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:46.341224 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1460/2924] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:49.344888 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fe380-f814-4509-8daf-a3ee83fc757e None None] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:49.364480 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:10:49.410567 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fe380-f814-4509-8daf-a3ee83fc757e None None] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:49.415897 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:49.416517 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:49.416827 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:49.417099 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] PATH_INFO: `/v3/users/1b683dd6f70349d5aa3cd10d59ee94ad` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:49.418212 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b683dd6f70349d5aa3cd10d59ee94ad)` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:49.420120 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:49.469027 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b683dd6f70349d5aa3cd10d59ee94ad'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:49.477319 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b683dd6f70349d5aa3cd10d59ee94ad'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:49.489505 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b683dd6f70349d5aa3cd10d59ee94ad'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:49.496198 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b683dd6f70349d5aa3cd10d59ee94ad'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:49.496893 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.notifications [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b683dd6f70349d5aa3cd10d59ee94ad'} {{(pid=81368) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:49.503696 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1465/2925] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Oct 3 16:10:49 2022] DELETE /identity/v3/users/1b683dd6f70349d5aa3cd10d59ee94ad => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:49.508164 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] REQUEST_METHOD: `POST` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:49.508651 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:49.508793 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:49.527858 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.auth.core [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] MFA Rules not processed for user `272aa82f64454c7aaff95459c4c29980`. Rule list: `[]` (Enabled: `True`). {{(pid=81367) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 16:10:49.529089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:49.588883 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1461/2926] 10.222.0.33 () {58 vars in 958 bytes} [Mon Oct 3 16:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 16:10:49.598864 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:49.600336 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:49.657991 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71454022-ac0a-4feb-81d4-b1bbe80b8b3d admin admin] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:49.661681 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d007a3d205345aebcc36a36d039da2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:49.662288 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:49.662558 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:49.662848 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:49.672234 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:49.673707 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:49.685719 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1466/2927] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:10:49.947271 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] Authenticating user token {{(pid=81367) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:10:49.949344 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.fernet_utils [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81367) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:10:49.969081 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dec5ae-eca8-4054-8a87-7b9c9ea24329 None None] Validating token access rules against request {{(pid=81367) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:10:49.976973 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '272aa82f64454c7aaff95459c4c29980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0727be3aa6194ac280b9115cbd69b50d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81367) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:10:49.979999 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:10:49.980309 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:10:49.980677 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] PATH_INFO: `/v3/projects/cfb36a6004e44735b8a085010b3ff7ab` {{(pid=81367) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:10:49.981763 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb36a6004e44735b8a085010b3ff7ab)` {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:10:49.983679 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] RBAC: Authorization granted {{(pid=81367) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:10:50.031697 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb36a6004e44735b8a085010b3ff7ab', 'request_id': 'req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:50.038841 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb36a6004e44735b8a085010b3ff7ab', 'request_id': 'req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:50.044441 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb36a6004e44735b8a085010b3ff7ab', 'request_id': 'req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9'} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:50.056089 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] The token cache is being invalidate because project cfb36a6004e44735b8a085010b3ff7ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81367) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 16:10:50.056374 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:50.057055 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG keystone.notifications [None req-1bd25a1d-f091-42e6-b7ba-d83fc06fe4b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81367) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 16:10:50.058542 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: [pid: 81367|app: 0|req: 1462/2928] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Oct 3 16:10:49 2022] DELETE /identity/v3/projects/cfb36a6004e44735b8a085010b3ff7ab => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 16:10:59.680131 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:11:00.052945 dv-cinder-844563-13-iscsi devstack@keystone.service[81367]: DEBUG dbcounter [-] [81367] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81367) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 16:12:19.730594 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] Authenticating user token {{(pid=81368) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 16:12:19.731994 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:12:19.800985 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca1b84-726f-4718-bf92-3cc7614daea1 service neutron] Validating token access rules against request {{(pid=81368) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 16:12:19.859488 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21b0a9-0e68-4e13-8d04-3ec5d67b5aca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92a197dc8b2749e08aa0d0612996cb1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78c46fd4a4864995b6630028c672c630', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81368) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 16:12:19.860091 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b0a9-0e68-4e13-8d04-3ec5d67b5aca service placement] REQUEST_METHOD: `GET` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 16:12:19.860371 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b0a9-0e68-4e13-8d04-3ec5d67b5aca service placement] SCRIPT_NAME: `/identity` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 16:12:19.860641 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21b0a9-0e68-4e13-8d04-3ec5d67b5aca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81368) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 16:12:19.863004 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.fernet_utils [None req-fd21b0a9-0e68-4e13-8d04-3ec5d67b5aca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81368) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 16:12:19.874138 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21b0a9-0e68-4e13-8d04-3ec5d67b5aca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 16:12:19.875714 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21b0a9-0e68-4e13-8d04-3ec5d67b5aca service placement] RBAC: Authorization granted {{(pid=81368) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 16:12:19.885752 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: [pid: 81368|app: 0|req: 1467/2929] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Oct 3 16:12:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 16:12:29.881390 dv-cinder-844563-13-iscsi devstack@keystone.service[81368]: DEBUG dbcounter [-] [81368] Writing DB stats keystone:SELECT=58 {{(pid=81368) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}